Methods, systems, and data structures for performing searches on three dimensional objects

ABSTRACT

Techniques are provided for searching on three dimensional ( 3 D) objects across large, distributed repositories of  3 D models.  3 D shapes are created for input to a search system; optionally user-defined similarity criterion is used, and search results are interactively navigated and feedback received for modifying the accuracy of the search results. Search input can also be given by picking  3 D models from a cluster map or by providing the orthographic views for the  3 D model. Feedback can be given by a searcher as to which models are similar and which are not. Various techniques adjust the search results according to the feedback given by the searcher and present the new search results to the searcher.

PRIORITY

[0001] The present application claims priority to U.S. ProvisionalApplication No. 60/442,373 filed on Jan. 25, 2003, entitled:“Architecture and Description for a 3D Shape Search System,” thedisclosure of which is hereby incorporated by reference.

TECHNICAL FIELD

[0002] Embodiments of the present invention relate generally to searchand retrieval, and more particularly to search and retrieval based onthree dimensional objects.

BACKGROUND INFORMATION

[0003] Search engines for text and images have been well developed overthe past decade. This invention deals with searching for threedimensional (3D) models from a repository of Computer Computer-AidedDesign (CAD) models.

[0004] CAD and Computer-Aided Manufacturing (CAM) software has grownfrom stand-alone environments towards network based collaborativeenvironments. Advances in 3D graphics hardware have contributed greatlyto the increasing use of 3D CAD models. Increased use of 3D geometrywill lead to internal design repositories which contain product relatedinformation such as CAD models and context-specific documents such asspecifications, manufacturing plans, etc. Typically, design repositoriesare analogous to distributed databases, spread over variousdepartments/people in a company.

[0005] Often designers are constrained by requirements of cost and time,making design reuse an attractive option. Design reuse necessitatessearching for past designs of parts and assemblies. Furthermore, asdesigners change within companies a lot of “knowledge” is lost with pastdesigners. Most of this “lost corporate knowledge” can be recovered backby searching for past designs. Clearly, reuse of past designs andknowledge from large design repositories is a key factor in compressingproduct development time and can deliver significant strategic advantageto a company.

[0006] Thus, there is a need to develop techniques that address thefollowing problems:

[0007] Design Reuse: Designs which can be reused by searching for pastdesigns in a company. The cluster map interface or 3D quick shapecreation interface which can be used to select or sketch the part mostsimilar to the one being designed.

[0008] Cost estimation: Design histories of the retrieved parts can bemade available thereby enabling the cost estimation process. A designercan look for the most similar part thereby reducing errors and savingtime in cost estimation

[0009] Assist in quotation process: Often, when a request for quotationis received, time is spent in searching for past designs. Thus, there isa need to reduce the time needed in responding to a Request For Quote(RFQ).

[0010] Part Classification System: Most companies employ people tospecifically classify parts in their database. This not only takes timebut is also dependent on human accuracy. Thus, there is a need tostreamline this process and classify parts based on desired metrics.

[0011] Reduce part duplication: There is a need, such that beforedesigning a part, designers can search for parts available in thedatabase to check if a similar part has already been designed.

[0012] Search engine for 3D shapes: Currently available 3D shape searchengines do not produce good results. Thus, there is a need to improveconventional 3d shape search engines.

[0013] Currently available database search techniques use keywords ornatural language based strategies. However in the case of 3D CADdatabases since the context is disconnected from the CAD model, it isdifficult to search for CAD models using keywords. Approaches such as akeyword search using the model filenames are also inefficient becausemost companies have a naming scheme that changes with time. Further,most designers tend to remember shapes better than the product contexts,making keyword based searches unattractive to users.

[0014] Moreover, there is a need to improve searching of 3D proteinstructures for applications such as drug discovery and docking.

SUMMARY OF THE INVENTION

[0015] Embodiments of the invention provide techniques for 3D search andretrieval. More specifically, and in some embodiments, 3D shapes(objects) are represented as graphical data structures and housed in oneor more searchable data stores. Each 3D shape or parts of a particular3D shape can be retrieved from the data store with feature vectors. Thefeature vectors include topology information and local geometricinformation associated with entities in the 3D shape or parts of the 3Dshape.

[0016] In some embodiments, a 3D shape or a part of a 3D shape can beused as a search request to the one or more searchable data stores. The3D shape or part embodied as a search request can be interactiveconstructed from scratch, selected by browsing a data store of 3D shapesor parts, or pasted or selected from an existing 3D shape or part.

[0017] In still more embodiments, an answer set returned from a 3Dsearch request can be organized into one or more related clusters, eachcluster includes one or more 3D shape or parts associated with aparticular portion of the answer set. The returned results and clusterscan be interactively rated for relevance or non relevance andresubmitted as a modified search request. The interactions and number ofresubmitted and modified search requests can continue until a searcheris satisfied with the results. Moreover, the relevancy information canbe associated with the searcher and used to intelligently alter anysubsequent search requests issued from the searcher.

BRIEF DESCRIPTION OF THE FIGURES

[0018]FIG. 1 is a diagram depicting a 3D shape search system.

[0019]FIG. 2 is a diagram depicting CAD and B-Rep Graph.

[0020]FIG. 3 is a diagram depicting interactions of a Module in ViewArchitecture (MVA).

[0021]FIG. 4 is a diagram depicting features of a MVA.

[0022]FIG. 5 is a diagram depicting a feature interface data structure.

[0023]FIG. 6 is a diagram depicting an example sketched plane tool.

[0024]FIG. 7 is a diagram depicting an example cutting plane tool.

[0025]FIG. 8 is a diagram depicting an example extrusion tool.

[0026]FIG. 9 is a diagram depicting portions of a data structure that iserased.

[0027]FIG. 10 is a diagram depicting example views of skeleton creationand modification.

[0028]FIG. 11 is a diagram depicting graphical views of example queryand search interfaces.

[0029]FIG. 12 is a diagram depicting an example of discretization in 2Ddiscrete space.

[0030]FIG. 13 is a diagram depicting types of voxel adjacencies in 3Ddiscrete space.

[0031]FIG. 14 is a diagram depicting example pseudo code for avoxelization algorithm.

[0032]FIG. 15 is a diagram depicting a 3D model showing face objects.

[0033]FIG. 16 is a diagram depicting face and facet data structures fora cube.

[0034]FIG. 17 is a diagram depicting face identifications stored in avoxel for a 3D model.

[0035]FIG. 18 is a diagram depicting an example shape and its skeleton.

[0036]FIG. 19 is a diagram depicting skeletonization.

[0037]FIG. 20 is a diagram depicting categorization of sample shapes.

[0038]FIG. 21 is a diagram depicting an example Euclidean structure withunwanted appendages marked.

[0039]FIG. 22 is a diagram depicting an example 3D voxel model FIG. 23is a diagram depicting example tubular parts.

[0040]FIG. 24 is a diagram depicting example internal cavities.

[0041]FIG. 25 is a diagram graphically depicting a technique for filinginternal cavities and example pseudo code for the same.

[0042]FIG. 26 is a diagram depicting classification of voxels based onvisibility.

[0043]FIG. 27 is a diagram depicting an example solid angle.

[0044]FIG. 28 is a diagram depicting an example 3×3×3 mask and itsassociated directions for an example shape.

[0045]FIG. 29 is a diagram graphically depicting an example Bresenhamalgorithm in 2D spaces.

[0046]FIG. 30 is a diagram depicting another view of the exampleBresenham algorithm of FIG. 29.

[0047]FIG. 31 is a diagram depicting an example wireframe technique.

[0048]FIG. 32 is a diagram depicting the preservation of topology in anexample wireframe model.

[0049]FIG. 33 is a diagram depicting a simplified wireframe model.

[0050]FIG. 34 is a diagram depicting various Levels Of Detail (LOD) fora simplified wireframe model.

[0051]FIG. 35 is a diagram depicting more views of various LOD for asimplified wireframe model.

[0052]FIG. 36 is a diagram depicting models that are visually similarbut still distinct.

[0053]FIG. 37 is a diagram depicting an example skeleton graph.

[0054]FIG. 38 is a diagram of example skeletal edges for local volumedistributions.

[0055]FIG. 39 is a diagram of an example CAD module with skeletalentities.

[0056]FIG. 40 is a diagram of example pseudo code for a marchingalgorithm.

[0057]FIG. 41 is a diagram depicting edge segmentation.

[0058]FIG. 42 is a diagram depicting the classification of loops.

[0059]FIG. 43 is a diagram graphically depicting volume basedreconstruction.

[0060]FIG. 44A is a diagram graphically depicting a view of theprocessing results associated with a marching algorithm.

[0061]FIG. 44B is a diagram graphically depicting another view of theprocessing results associated with a marching algorithm.

[0062]FIG. 44C is a diagram graphically depicting another view of theprocessing results associated with a marching algorithm.

[0063]FIG. 45 is a diagram depicting an example skeleton model.

[0064]FIG. 46 is a diagram depicting a cleaned-up skeleton model.

[0065]FIG. 47 is a diagram of a flowchart for the data flow associatedwith a marching algorithm.

[0066]FIG. 48 is a diagram depicting an association graph technique.

[0067]FIG. 49 is a diagram depicting a tree-based multidimensionalindex.

[0068]FIG. 50 is a diagram depicting nodes in a tree-basedmultidimensional index.

[0069]FIG. 51A is an example diagram of a tree-based multidimensionalindex data structure.

[0070]FIG. 51B is another example diagram of a tree-basedmultidimensional index data structure.

[0071]FIG. 52 is a diagram depicting feature vectors.

[0072]FIG. 53 is a diagram depicting models as points in feature space.

[0073]FIG. 54 is a diagram depicting a visual interface for a partcluster map.

[0074]FIG. 55 is a diagram depicting a Self Organizing Map (SOM) datastructure.

[0075]FIG. 56 is a diagram depicting a cluster map interface.

[0076]FIG. 57 is a diagram depicting a cluster map with automaticclusters for browsing.

[0077]FIG. 58 is a diagram depicting a cluster map after performing asimilarity query.

[0078]FIG. 59 is a diagram of a flowchart depicting a method forrelevance feedback.

[0079]FIG. 60 is a diagram of a technique for customization based onrelevance feedback.

[0080]FIG. 61 is a diagram of a matching technique between data storevectors and query vectors.

[0081]FIG. 62 is a diagram depicting an example method for a multi-stepsearch process.

[0082]FIG. 63 is a diagram of example search results produced from amulti-step search process.

DETAILED DESCRIPTION

[0083] In various embodiments of the invention the term “user” ispresented. A user is a searcher and a searcher can be any entity that isrepresented in a computer accessible medium, such as an agent, aservice, an application, a physical user, and others. Moreover, althoughvarious embodiments describe the use of a database, it is to beunderstood that the data base can be any data store or collections ofdata stores, such as electronic filed, directories, relationaldatabases, object oriented databases, data warehouses, and the like.Finally, a 3D object is an electronic representation of a 3D shape orportions of a 3D shape, in some cases some 3D objects can be viewed as3D models.

[0084]FIG. 1 is a diagram of a 3D shape search system 100. The system100 provides a platform for users to search for similar models fromlarge repositories of 3D models. Complicated 3D models are reduced to asimpler representation called a skeleton. Important distinguishingfeatures from the 3D model and the skeleton are extracted and stored asa feature vector in a database. Finally the database is searched and theresults are shown as a clustered output. During the process ofconverting a complicated 3D model to a simpler skeletal form(“skeletonization” from hereon), searching the skeletal representationand clustering the output, a lot of intermediate steps occur as shown inFIG. 1.

[0085] The search system 100 is adapted to include:

[0086] Insensitivity to minor changes in shapes

[0087] Combination of geometry and topology

[0088] Multi-level and multi category representation of shapes

[0089] Accuracy and computational efficiency

[0090] Quick, user-customized query construction

[0091] Dynamic Query Creation and Modification

[0092] The 3D search system 100 provides users with an intelligentinterface. A 3D skeleton modeling interface is implemented such that itis capable of not only letting the user sketch the skeleton of the modeland submit it as a query, but also enabling the user to dynamicallymodify the skeleton of the retrieved models to further improve hischances of finding the intended part. The system 100 is the first of itskind to allow creation and 3D manipulation of a 3D skeleton. The changesmade on the topology of the skeleton of the model will make the system100 redo the search while changes only to the geometry of the skeletonwill only make it reorder the retrieval. Through this interaction, thequery is expected to be more close to the user's intention and thesystem 100 can improve its performance more efficiently.

[0093] Local Query

[0094] Conventionally, the only type of query that a user could make wasto ask for similar CAD models, by specifying an input CAD model, butthis will not best satisfy the user's requirements. For example, theuser may want to assign more importance (or emphasis) to a specificfeature or his/her main focus for search might be a local feature in theCAD model rather than the whole model itself. Such flexibility is notpossible with the past works and their interfaces. Each search isdifferent and the user interface should be flexible enough to handle thedifferent requirements. Further, better query processing is needed toaccommodate the various requests. The system 100 is designed to bettermeet user requirements and to improve query processing needed to achievethe same.

[0095] Flexibility/Generality of the Search Criteria

[0096] Past techniques have focused generally on comparing two shapes.But there are a host of issues specific to an application domain. Onemain issue is the interpretation of the similarity of CAD models. Forexample, with respect to the manufacturing domain, the various machiningfeatures in a CAD model assume greater importance. Computing any shapedescriptor number, as in the probabilistic shape distribution methodsmay not be customizable to the specific domains and hence cannot takeserve the user's needs properly.

[0097] Similarity Measure

[0098] Finally the user may want to quantify the shape similaritybetween the input and the results. This is possible to an extent in theshape function methods. But the definition for similarity itself is userdependent. Hence the interface should attempt to learn the user'sperception of similarity of shapes. This is not possible with theexisting research techniques.

[0099] Shape: A combination of Geometry and Topology

[0100] The shape of any 3D model is a combination of topology and thegeometry of its constituent entities. While geometry is the quantitativemeasure of an entity's shape, size and location, topology is relationalinformation between geometric entities of the model. Hence, anyrealistic shape representation should reflect these two ideassufficiently. Separate representations of these characteristics willallow us to have a hierarchical search strategy for similar shapes.

[0101] The approaches in prior art represents shape as either one ofthese quantities or as a ‘composite measure’ which has no distinctionbetween the geometry and topology. For example, shape descriptors suchas moment invariants and shape distributions are ‘composite measures’ ofthe shape without any distinction between model geometry and topology.No particular approach can be described as generic unless it allowssearch based on the intent of the user. Representing 3D models byrelational structures such as graphs/trees achieves a lot in maintainingthe distinction between geometry and topology. However, graphrepresentation alone may not be able to solve the problem of capturingthe intent of the user. Further research is therefore needed to make useof the geometric information efficiently during the search/indexingprocess.

[0102] Graph Data Structure instead of Tree Data Structure

[0103] Herein novel methods, systems, and data structures are describedthat explain why conventional techniques are not applicable for certainenvironments, such as mechanical engineering and hence distinguish thisinvention from past approaches.

[0104] Conventional skeleton/shock graph based 3D shape matching systemswere developed for multimedia retrieval where the models are general 3Dobjects such as Aircraft (full, approximate 3D model), Cars, Fruits,etc., but are not domain-specific. Some conventional approaches havebeen tested on models without holes in their topology, because solidshapes having a set of surfaces forming a closed hole cannot be capturedby conventional tree structures. However, such closed loop entities areencountered often and are significant in the mechanical engineeringdomain, which call for Graph structures for their representation.Furthermore, other conventional approaches are not amenable forcomparing local geometry or for partial matching. The system 100 andother novell techniques presented herein convert skeletons into a Graphdata structure and captures the loops in solid shape.

[0105] Multi-Step Search Strategy

[0106] Prior technqiues only allows a single query and allows norefinement of that single query. In addition, that search is based on apredefined fixed set of feature vectors which may not guarantee theretrieval of relevant shapes. System 100 allows a user to filter searchresults successively using a mutli-step search interface. Although thisapproach may not guarantee the retrieval of relevant shapes, itincreases the chances of retrieving relevant shapes and thus offerssubstanitail improvement over conventional approaches. A large number offeatures including moment invariants, volume, aspect ratio, andeigenvalues are provided to the user to assist in posing multi-stepqueries.

[0107] Computational Efficiency

[0108] Computational Efficiency controls the ability of the system tohandle huge volumes. Some of conventional methods concerning B-Repgraphs are inefficient as B-Rep graphs are huge in size for simple 3Dmodels. Exact graph matching algorithms are NP-hard. So comparing agiven CAD model to the millions of parts in the database using the B-Repgraph methods seems impossible even with supercomputers.

[0109]FIG. 2 is a diagram 200 of a CAD and B-Rep Graph. Diagram 200shows a simple part and its corresponding B-Rep graph. Most conventionalapproaches developed for 3D model comparison fail to use domainknowledge. Although shock/skeletal graph based approaches are a stepforward in this direction, the information present in the models is notproperly explored to reduce the complexity of the graph/tree matchingproblems. This invention uses domain-specific knowledge including thetype of entity (loops/edges) and local geometry and volume informationfor reducing the complexity of search, and in developing adomain-specific search system.

[0110] Disk I/O Efficiency

[0111] Conventional efficiency in terms of disk I/O is far fromsatisfactory. When the number of CAD models becomes very large, the diskI/O time will be a large part in the query time. For example, supposingthat a database has one million models, and 15 milliseconds for a diskI/O time, and each CAD model occupies one disk page (in some research,the disk files resulting from the matches take, on an average, 1 MB ofdisk space), it takes 4 hours for a supplely scan over the recodes inthe database. Therefore, it is important to index the data to speed upthe search. However, the index structure of current database systems,such as B+tree which has made great success, is not eligible for thesemulti-dimensional data. Therefore, new indexing techniques to filterunnecessary disk I/O is of great importance. This problem, however, hasnot gained enough emphasis. One of the reasons for this is that most ofthe current 3D shape searches are still focusing on searching among verysmall number of CAD models, because of the inefficiency of theirapproaches.

[0112] System Processing

[0113] The process of searching for a 3D model can be described by thefollowing sequence of operations:

[0114] 1. User interface side:

[0115] 1.1. Query Sketch Interface

[0116] Sketch the required shape in 3D OR

[0117] Sketch the skeleton (stick figure) of the required 3D shape OR

[0118] Choose a model from the model cluster OR

[0119] Choose a 3D solid model from the local hard drive OR

[0120] Submit or sketch orthographic views for a 3D model

[0121] 1.2. Feature Vector Interface

[0122] Choose required combination of Feature vectors for search andclustering

[0123] 2. Server or Service Processing:

[0124] 2.1. Voxelization

[0125] Convert the 3D model into a neutral discrete voxel representation

[0126] Extract and store 3D model and voxel feature vectors

[0127] 2.2. Skeletonization

[0128] Depending on the detected topology of the voxel model, performone of the three skeletonization operations (details are presentedlater):

[0129] 2.2.1. Prismatic Skeletonization

[0130] 2.2.2. Tubular Skeletonization

[0131] Simplified Wireframe

[0132] Extract and store skeleton feature vectors

[0133] 2.3. Graph Creation

[0134] Detect entities from the skeleton depending on skeleton type(loops, edges or surfaces)

[0135] Extract and store graph feature vectors

[0136] 2.4. Conversion of 2D Drawings into 3D Models

[0137] Noise removal

[0138] Dimension and title block extraction and removal

[0139] 2D Thinning or 2D Skeletonization

[0140] Edge and entity recognition

[0141] Obtain continuous closed segment loops in orthographic views

[0142] Identify interior disjoint loops

[0143] Skeletonization of non-disjoint loops using MAT

[0144] Conversion of skeleton into 3D skeleton

[0145] Convert 3D model into skeleton using any of the aboveskeletonization methods

[0146] 3. Database, Search and Clustering:

[0147] 3.1. Graphs Comparison

[0148] Either by graph/sub-graph isomorphism, association graphtechniques or probabilistic graph matching techniques.

[0149] 3.2. Create Multidimensional Index Structure

[0150] Use R−tree, R+tree, KD−tree, X−tree or any other spacepartitioning techniques to index the database.

[0151] 3.3. Feature Vector Comparison

[0152] Based on user preferences as in 1.2 above. If not use a defaultcombination of feature vectors

[0153] 3.4. Cluster Models in Database

[0154] Use self organizing map (SOM) or any other hierarchicalclustering techniques

[0155] 3.5. Present Results to User Through Cluster Map Display

[0156] Pick a representative model from each cluster and display thepart image through a user-friendly interface

[0157] Allow picking of models for browsing

[0158] 3.6. Relevance Feedback

[0159] The user picks parts that are relevant and non-relevant

[0160] The results are sent back to the database through the server

[0161] Cluster map is updated with user relevance feedback results

[0162] Results are displayed back to the user

[0163] 3.7. Learning Agents

[0164] Learning algorithms, such as neural networks, “learn” userrelevance feedback preferences and stores them in a user profile

[0165] 3.8. Multi Step Search

[0166] Allow user to use different feature vectors and skeletal graphsfor refining search results in a sequential or step-by-step manner.Similarity comparison is same as those described in 3.1 and 3.3 above.

[0167] The architecture of the system 100 can be best described by at ahigh level in FIG. 1. The sections below, describe embodiments of theinvention in greater detail.

[0168] 1. User Interface Side

[0169] Referring again to FIG. 1, the user interface essentiallyconsists of three interfaces, namely, Query-by-Sketch (QBS),Query-by-Example (QBE), and Feature vector interface. The QBS interfaceimplements two different versions (i) 3D solid model sketching interfaceand (ii) 3D skeleton sketching interface. The QBE interface consists ofa (i) Cluster Map Interface, (ii) Local Hard Drive and a (iii) 2DDrawing Interface.

[0170] 1.1. Query Interface

[0171] The following subsections describe the various methods andmodules that are used to construct a query in our system (QBS and QBE)either directly as a 3D solid model, as a 3D skeleton model or from the2D views of a 3D model.

[0172] 1.1.1. 3D Solid Model Sketching

[0173] Description of the Module:

[0174] This module provides a web-based and platform independent systemto create approximate geometric shapes as quickly as possible. It uses ahybrid approach to shape creation, for instance, the use of featurebased and sketch based solid modeling. The sketch interface interpretsmouse strokes and converts these discrete strokes into primitiveentities like, line, point, arc using a vectorization algorithm. Theconverted sketch can be then used to form solid geometry. User can alsouse the predefined shape features such as block, cylinder, hole, asprimitive building blocks. The faces of these features can then be usedas a canvass for further sketching.

[0175] The system 100 operates as follows:

[0176] User starts new session after logging into the shape searchwebsite.

[0177] He uploads or opens already existing CAD file.

[0178] Alternate ways to start modeling is by dragging and droppingpre-created solid shape features like block, cylinder or by sketchingapproximate sketch profile by dragging the mouse or pen.

[0179] If a sketch is created, it is sent to server for extrusion. Thiscan be achieved either by using separate buttons or through mouse or penstroke indicating extrusion.

[0180] Server receives the command and accordingly converts the sketchinto an extruded solid and sends back the faceted object to the client.

[0181] Newly created solid is viewed in the client window.

[0182] User selects a face of the solid and starts sketching on it.

[0183] The dragged predefined shape features can be edited to obtain theapproximate shapes. This editing involves changing the dimensions orpositions of the features, copying and pasting the features, deleting orsuppressing the features etc.

[0184] Due to approximate geometric shapes there might be over hangingor protruded portions of the solids which are not needed.

[0185] The user will then use a 3D eraser to remove unwanted part.

[0186] Most of the computationally intensive functionalities, e.g. solidmodeling, feature creation and modification, are transferred to theserver in order to reduce the size of the clients.

[0187] Module Architecture:

[0188]FIG. 3 is a diagram 300 of interactions occurring with aModule-in-View Architecture (MVA). The proposed interface is based onToolCAD architecture. It consists of Model-View-Controller (MVC)architecture. This forms the core of the module.

[0189] Model—Application object. The Geometry object in this case. Itcontains the details of the geometry being created. Geometry in thiscase consists of created solid shapes.

[0190] View—The screen representation of the Model. Magician libraryused to display the faceted geometry objects with their correspondingrelative transformations.

[0191] Controller—The way the user interface reacts to the user input.

[0192] The Magician library used to capture the user input like picking,dragging, etc.

[0193]FIG. 4 is a diagram 400 of features in MVA 300. In this casediagram 400 holds a list of features and sketches created. Features arepredefined objects which hold the data like, list of faces, edges withit. Sketches when extruded convert to features. These are same aspredefined features only in this case these hold pointer to theunderlying sketch. Diagram 400 consists of the faceted data that isshown to the user. A user can interact with this data and change it.

[0194]FIG. 5 is a diagram of a feature interface object 500. This mayrepresent one more steps into a hierarchy where model can withheldmultiple products. Each product will have separate Geometry object whichin turn will hold the features created or predefined. Geometry will aswell hold the faceted representation of the solid model formed afterBoolean operations are performed on the features. This faceted structure500 is used by a View object in order to display it to the user.

[0195] Feature Module: Features are predefined 3D shapes which can bedirectly used for creating solid model. Many times users want to havesame shapes again and again in the model. The most common examples canbe rectangular solid blocks, cylinders, holes, slots etc. To improve thespeed of geometry creation such shapes are kept ready to deploy into theworkspace. These shapes have default size when first put in theworkspace. However each face of the feature will have handles formanipulating the size and position of the feature. When a feature isselected, these handles are displayed and user can drag these handlesand change the size of the feature interactively. The features arecreated using ACIS solid modeler which recites on the server and thenfaceted to form a list of faces. Edges of these faces are extracted andare stored with each feature. All features are represented by just listof edges. So when a user selects a feature, it is shown in wire-framemode. However actual 3D model of the feature is stored on server sidewhich is then used to perform Boolean operations with all previouslycreated shapes to get the final shape model. Each sketched and extrudedshape is also a user created feature and can be saved separately forlater use. Thus the interface is quite much customizable. These usercreated features will hold a pointer to basic underlying sketch fromwhich these are created.

[0196] Sketch Module: After the user defines a sketching plane, thesketching begins by automatically rotating the sketch plane to make itparallel to the screen. The user then sketches an outline or a profile.The profile is then interpreted into vectors by a vectorizationalgorithm, such as, and by way of example only, the vectorizationalgorithm described by Dov Dori and Liu Wenyin. These vectors form theprofile of the wire that can be extruded to form a solid.

[0197] Tools for shape creation: For simplifying the process, some toolsare optionally provided. These tools can be:

[0198] Sketch Plane: User chooses any face of the solid shape to createsketch plane in either of the following ways. An example sketch planetool 600 is shown in FIG. 6. This tool can perform the followingprocessing:

[0199] 1. Offset to the face of solid already created.

[0200] 2. Choosing three points to pass a plane through it

[0201] 3. Choosing any edge of existing solid to create plane throughit.

[0202] 4. Rotating a sketch plane around any edge of solid.

[0203] Cutting Plane: Cutting plane is similar to sketching plane;however it is used interactively by the user for creating a cut into thesolid. The plane can be any sketched profile. An example cutting planetool 700 is shown in FIG. 7. FIG. 7 shows the working of the cuttingplane tool 700. This tool 700 uses functionality of face dragging andthen uses ACIS to finally create a cut into a solid.

[0204] Extrusion through Digital Pen or Mouse strokes: This function isimportant for creating extrusion from 2D shape profile. It is more thanintuitive that once the sketch loop is finished, a user will use samedevice that is the mouse or digital pen to extrude the profile. So afteruser finishes the loop, user prompts for extrusion through next mousestroke or drag and interactively specifies the depth till which he/shewants to extrude the profile. This is more similar to using cuttingplane. Example of an extrusion tool 800 is shown in FIG. 8.

[0205] 3D Eraser: This function is used to remove hanging parts of thesolid features. For this, a voxelized model of the solid shape is used.As shown in the example 3D eraser tool 900 of FIG. 9, a user will choosethe voxels to be removed by selecting them and then when he is finished,Boolean subtraction of selected voxels with the solid model is performedand the result is displayed to the user. User may or may not see theunderlying voxel model. This tool 900 is used to remove anyinconsistencies in the model.

[0206] 1.1.2. 3D Skeleton Creation and Modification Interface

[0207] A 3D skeleton modeling interface is implemented such that it iscapable of not only letting the user sketch the skeleton of the modeland submit it as a query, but also enabling the user to dynamicallymodify the skeleton of the retrieved models. The changes made on thetopology of the model will make the system redo the search while changesonly on the geometry will make it reorder the retrieval results. Throughthis interaction, the query is expected to be more close to the user'sintention and the system can improve its performance more efficiently.

[0208] This interface is initiated by the fact that the user may lacksuitable examples for the query. The system enables the user toconstruct the 3D skeleton by assemble the primitive shapes first andthen manipulate the shapes in 3D space (see FIG. 10 a diagram 1000 of aview of 3D shape manipulation). The resulted “skeleton” includes theessential topology and geometry information just like the skeleton fileobtained from the skeletonization. Currently the primitive shapesinclude edge such as straight line and curve, and loop such as circleand rectangle. The 3D manipulation annotations include rotation,panning, scale and zooming which are designed for both the individualshape primitive and the global assembly. The interface also allows theuser to delete a selected shape, clear current configuration, import askeleton graph file to visualize it, and submit the constructed skeletonas a query for searching similar shapes.

[0209] 1.1.3. Cluster Map Interface

[0210] See section 3.5 described herein and below.

[0211] 1.1.4. Local Hard Drive Interface

[0212] An option is given to the user to select a CAD file that hasalready been generated. The file can be anywhere where the user haspermissions to access the file. The file can be in any CAD formatsupported by our invention. A standard pop up window is provided to theuser to select the CAD file.

[0213] 1.1.5. 2D Drawing Interface

[0214] Users are provided options to send in their scanned drawings tothe server.

[0215] 1.2. Feature Vector Interface

[0216] The current invention includes techniques for customization ofthe feature vectors in partial or full by the user to describe hissearch intention accurately as shown in diagram 1100 of FIG. 11. Thesearch query is executed in multiple levels/stages to get betterresults. This interface allows the user to assign the different featurevector(s) to these various tiers in the search process. This helps theuser to describe his search intent more accurately. The user can furtherreassign the weights for these feature vector(s) as applicable toimprove his search. By engaging the various options in the interface caninstruct the system to do an exact or partial search. A partial searchquery involves sub graph isomorphism and this is dealt in the skeletalgraph level. The user can also put into use the neural network to getefficient query results. The user can set his search preferences and canalso store it with his profiles. A user can have multiple profilesdefined for various search situations as per his frequent requirements.This interface also allows the user to manage these profiles and also toprovide relevance feedback to the system.

[0217] 2. Server or Service Side

[0218] 2.1. Voxelization

[0219] Voxelization is the process of converting 3D geometric objectsfrom their continuous geometric representation into a set of voxels thatbest approximates the continuous object in a 3D discrete space. The 3Ddiscrete space is a set of integral grid points in 3D Euclidean spacedefined by their Cartesian coordinates (x,y,z). A voxel is the unitcubic volume centered at the integral grid point. The voxel value ismapped into {0,1}: the voxels assigned “1” are called the “black” voxelsrepresenting opaque objects, and those assigned “0” are the “white”voxels representing the transparent background. FIG. 12 shows a diagram1200 of an example for discretization in 2D space.

[0220] Two voxels are “26-adjacent” if they share a vertex, an edge, ora face (see FIG. 12). Every voxel has 26 such adjacent voxels—eightshare a vertex (corner) with the center voxel, twelve share an edge, andsix share a face. Face-sharing voxels are defined as “6-adjacent”, andedge-sharing and face-sharing voxels are defined as “18-adjacent”.Conventional approaches for voxelization use a polygonal model, aparametric curve or an implicit surface as input. Scan filling orrecursive subdivision algorithms are used for voxelization. Prior tovoxelization, the model is normalized with respect to translation andscaling. The scale factor is stored in the database for futurecomputations.

[0221] In one embodiment, an ACIS solid modeling kernel for voxelizationof a 3D model is used. The input to the ACIS kernel is a B-Rep model.FIG. 13 is a diagram 1300 depicting example pseudo code for avoxelization algorithm. In one embodiment, a bounding box for the 3Dmodel is constructed and a voxel size chosen depending on the minimumbounding box dimension or use a user specified voxel size. The discretebounding box dimensions are then calculated. All faces and edges in the3D model are assigned faceIDs and edgeIDs respectively and stacks arecreated for each face and each edge. The face and edge stacks can beindependent or dependent of each other depending on various programconditions, memory etc. The 3D model is looped through in increments ofthe voxel size and the intersection of every voxel with the model ischecked. A ‘1’ is appended to the text file at the position containing avoxel and a ‘0’ are every position not containing a voxel. For allpositions containing a voxel, get all the faces and edges intersectingwith the model and store the faceIDs and edgeIDs of the correspondingfaces and edges in the voxel under consideration. For the faces andedges intersecting with the voxel, store the voxel under considerationin the face and edge stacks. Thus we now have references for a voxelfrom a face/edge and vice-versa. This can be used as input to thesimplified wireframe algorithms described elsewhere in this document. Itis important to note that voxelization for a polygonal model can also beperformed. The process for voxelization of a polygonal model ispresented in the next section.

[0222] A 3D model consists of a list of Face objects. A Face objectrepresents a face of the resulting three-dimensional solid. A Faceconsists of an ordered list of the vertices of the face and a list ofFacet objects as shown in the example 3D model 1400 presented in FIG.14. The Facet list contains the list of triangles generated bytriangulating the face of the solid. This structure is better thanstoring only a list of facet triangles as it can show the correct wireframe view of the solid model. This structure also maintains theinformation of all the triangles belonging to a face that makes itpossible to pick a face at the client-side. FIG. 15 shows an example ofthis data structure for a cube 1500. The cube 1500 consists of 6 Faceobject. Face objects consist of points and Facet objects, e.g., Face 2consists of 4 points (A, B, C, and D) and 2 Facet objects. One of theFacet objects, as shown in FIG. 15, stores the three points: A, B, andC, normal to the Facet, and the Face number to which the Facet belongs.Since one knows the mapping between Facet and Face, one can performvoxelization of a polygonal model and store Face IDs into each voxel asdescribed in the sample pseudo code 1300 of FIG. 13 above.

[0223] Significant differences between the presented examplevoxelization algorithm and previous approaches are listed below:

[0224] Parallelization: Since voxelization is a simple domaindecomposition problem, it is amenable to parallel computing. Each slaveprocessor can be assigned a domain range and the master processor canassemble the results together.

[0225] Voxel size: A size range for voxelization of 3D engineeringmodels is developed. It is seen that ideal voxel sizes for mostengineering shapes falls in the range of {fraction (1/32)}-{fraction(1/64)} times the minimum bounding box dimension. The condition for avoxel size being “ideal” is that the major features in a part arecaptured in the voxel model; i.e. very small features may be lost duringvoxelization because the feature size is smaller than the voxel size.

[0226] Adaptive voxelization: Adaptive voxelization for a 3D model isperformed. Adaptive voxelization is the iterative process of voxelizinga 3D model, evaluating the voxel model for exactness with the 3D modeland finally re-voxelizing with a voxel size lesser than the earliervoxel size until an acceptable voxel model has been attained.

[0227] Storing face/edge information in voxel: The 3D model storesFace/Edge information in every surface voxel. An example of Face IDsbeing stored in voxels is shown in diagram 1600 of FIG. 16. Prior tovoxelization, Face and Edge IDs are assigned to every face and edge inthe 3D model. As voxelization is performed face/edge intersections forevery voxel are checked and face/edge IDs are stored in intersectingvoxels. The process of edge ID storage is similar to the process ofstoring face IDs shown in FIG. 16. The Face/Edge ID information is usedin the simplified wireframe as described in greater detail later in thisinvention.

[0228] Voxelization using a solid modeling kernel: With variousembodiments of this invention, voxelization is preformed using the ACISsolid modeling kernel as opposed to performing voxelization using any ofthe available algorithms available. This is important because none ofthe conventionally available algorithms are capable of storing face andedge IDs in voxel models.

[0229] There are a variety of other uses of the present voxelizationalgorithm, such as:

[0230] Distance Volume Generation: A distance volume is representedwithin a volume dataset. A particular value stored in a particular voxelrepresents the shortest distance to a surface of the 3D model that itrepresents. A distance volume representation is an important dataset formany graphics applications, most notably in the manufacturingvisualization domain. A distance volume is a natural consequence ofusing ACIS for voxelization, since one can calculate distances from thesurface of the model and store it in the voxel data structure. Someapplications of using distance volumes are CSG surface evaluations,offset surface generations, determination of regions of maximum orminimum thickness in net-shape manufacturing, and 3-D model morphing.

[0231] 2.2. Skeletonization

[0232] 2.2.1. Definition of Skeletonization

[0233] In various embodiments of the invention, a skeletalrepresentation (hereinafter “skeleton”) is defined as: “the minimumrepresentation of the major shape of a part in 3D”. In other words, askeleton is a simplified representation that neglects the “minor”features in a shape such as small radii and holes while still retainingthe “major” features and preserving “major” topology relationships inthe part. Skeletonization is any process that generates such a skeleton.

[0234] A skeleton is:

[0235] a. Centered from the boundary

[0236] b. Preserves connectivity

[0237] c. Free from boundary noise

[0238] d. Rotationally invariant

[0239] e. Computationally simple

[0240] 2.2.2. Mathematical Model of a Skeleton and Skeletonization

[0241] Formal Definition of a Skeleton

[0242] A skeleton is a tuple S=(G, T) where:

[0243] G={g ₁,g₂, . . . ,g_(n)} is the finite set of geometric entitiescomposing a skeleton, and

[0244] T is a n×n adjacency matrix that defines the topologicalconnectivity between elements of G.

[0245] For a skeleton to exist, G is defined as the set:

[0246] G=E∪L∪S∪H where:

[0247] E={e₁,e₂, . . . ,e_(p)} is the set of edges,

[0248] L={l₁,l₂, . . . ,l_(q)}is the set of loops,

[0249] S={s₁,s₂, . . . ,s_(r)} is the set of surfaces, and

[0250] H={h₁,h₂, . . . ,h_(s)} is the set of holes

[0251] However, the sets E, L, S and H need not all be non-zero for aninstance of S.

[0252] Matrix T is defined as:

[0253] T=[t_(ij)]_(n×n) where:

[0254] t_(ij)=N if g_(i) is adjacent to g_(j)

[0255] 0 if not

[0256] and N is a nonzero number describing the connectivity betweenentities g_(i) and g_(j).

[0257] For a skeleton to exist, both G and T must be non-zero.

[0258] Formal Definition of a Geometric Feature

[0259] A geometric feature γ in a shape ψ is defined as a set ofgeometric entities e_(i)(i=1 . . . n) that reduce to just one geometricentity in a skeleton S. In some cases, it may be possible that theentire shape is a single geometric feature.

[0260] In example shape 1700 shown in FIG. 17, there are two geometricfeatures present in shape ψ even though the number of geometric entitiesin T is more than two. The skeleton S has two geometric entities—a loopand an edge.

[0261] Formal Definition of a Shape

[0262] A shape is a tuple ψ=(γ,τ) where: γ={γ₁,γ₂, . . . ,γ_(m)} is theset of geometric features composing a shape, and

[0263] τ is a m×m adjacency matrix that defines the topologicalconnectivity between elements of γ.

[0264] Matrix τ=[τ_(uv)]_(m×m) where:

[0265] τ_(ij)=1 if γ_(u) is adjacent to γ_(v)

[0266] 0 if not

[0267] For a shape to exist, both γ and τ must be non-zero.

[0268] Formal Definition of Skeletonization

[0269] Skeletonization is defined as a mapping O: ψ_(k)→S_(k)−□ where:

[0270] S_(k) is the skeleton produced by subjecting a shape ψ_(k) to theskeletonization operator O and, □ is a noise factor removed duringskeletonization.

[0271] It is easy to see that in the present context skeletonization isa many-to-one mapping as shown in FIG. 18, an example skeletonizationmapping.

[0272] 2.2.3. Categorization of Shapes

[0273] Engineering shapes can be categorized into the followingcategories:

[0274] i. Solid: Solid-like shapes usually have high material usagecompared to their size. They usually have non-constant wall thicknessand have few features. Solid shapes are typically manufactured bymachining or forging. Very few solid parts are manufactured by net-shapeprocesses such as molding or casting.

[0275] ii. Shell: Shell-like shapes usually have minimal material usagecompared to their size. They usually have constant wall thickness andare simple shapes. Solid shapes are usually manufactured by sheet metalor net shape processes such as molding or casting. Tubular shapes alsocome under the category of shell-like shapes.

[0276] iii. Hybrid: Hybrid shapes, as the name suggests are acombination of the above two shapes. Some parts of the shape havesolid-like features and some parts have shell-like features. They areusually manufactured by molding or casting.

[0277] Examples of each type of shape are shown in FIG. 19.

[0278] 2.2.4. Skeletonization Methods

[0279] It is observed that skeletonization methods for the above threecategories of shapes are different from each other. FIG. 19 also showsthe skeletonization methods for different categories of shapes. Forexample, a prismatic skeletonization method for solid shapes, a tubularskeletonization method for the special category of tubular shapes (whichare shell-like) and a simplified wireframe method for the remainingshapes (which may be shell-like or hybrid). Any combination ofprismatic, tubular and simplified wireframe skeletons could also beformed as a skeleton with appropriate user interactions.

[0280] 2.2.4.1. Prismatic Skeletonization

[0281] The concept of a skeleton is as a unique descriptor of shape.Thus, for some embodiments of this invention, a skeleton therebydescribes a class of shapes that may have similar engineeringcharacteristics. Various present embodiments search for similarskeletons, which will lead to similar shapes. A related area in imageprocessing employs a skeletonization process for simplification ofcomplex pixel (2D) or voxel (3D) discrete geometry into a wire (1D)representation. Three approaches for generating skeletons from 3Dgeometry are:

[0282] i. Distance transformation: Distance transforms which have beenwidely used for skeletonization. They are fast and easy to compute.However, skeletons from distance transforms require significant noisefiltering and sometimes have topological instability. i.e. they do notpreserve topology.

[0283] ii. Euclidean skeletons: Euclidean skeletons are also calledmedial axis transforms. They are computationally complicated andgenerate unwanted appendages during skeletonization. Because of thesereasons, Euclidean skeletons will not be preferably used with variousembodiments this invention. However, an additional pruning process canbe developed for them to be used with the teachings of this invention.An example of a Euclidean skeleton is shown in FIG. 20.

[0284] iii. Thinning algorithms: Thinning algorithms are widely used inthe medical imaging domain. Thinning is a process that reduces a voxelmodel to a skeletal representation that is a voxel thick. Voxels areremoved after checking conditions for preserving topology. Sixdirections are typically defined for checking topological connectivity:UP, DOWN, EAST, WEST, NORTH and SOUTH. Two types of thinning algorithmsconventionally exist—Serial and Parallel.

[0285] Serial algorithms, as the name suggests, perform thinning byremoving voxels from any one of the six directions independently.Parallel algorithms remove voxels from all six directions concurrently.In practice any of these algorithms can be used for thinning. In thisinvention, any parallel thinning algorithm that reduces a 3D voxel modelto a medial axis rather than a medial surface can be used. Examples of a3D voxel model, its medial surface and its medial axis are shown in FIG.21. Thinning is the one used method of skeletonization in thisinvention, since it best balances the requirements of topologicalconnectivity, stability under small changes, and computationalefficiency. Furthermore, a combination of distance transforms andthinning can be used for obtaining a prismatic skeleton.

[0286] 2.2.4.2. Tubular Skeletonization

[0287] Tubular model is defined as a 3D geometry model which includesone or more tube structure. Because a thinning algorithm is based onprismatic parts, thinning a 3D tubular part can sometimes cause anunexpected result. Thinning algorithms do not work well when a part iscomposed of multiple tubes as in the examples of tubular parts of FIG.22. In this case, a pre-processing step called Filling Internal CavityVolume (FICV) is applied to the part voxels to convert tubular partvoxels to prismatic part voxels.

[0288]FIG. 23 shows the volume that should be filled to convert tubularpart voxels to prismatic part voxels. The shaded volume in FIG. 23 iscalled tubular cavity volume. The basic concept of FICV is filling thetubular cavity volume. To fill the tubular cavity volume, it is requiredto identify which portion of the cavity volume is to be filled among allcavity volumes in the boundary volume of the model. The cavity volumecan be obtained through the voxel model. After voxelization, each voxelhas a “1” value if the voxel is located in the model and has a “0” valueif the voxel is located outside of the model. Image2 in FIG. 24 showsthe voxelization result and image4 in FIG. 24 shows the inversevoxelization result which represents cavity volume. However, the cavityvolume does not always mean tubular cavity volume because the cavityvolume also includes some redundant volume in the boundary volume. Onthe basis of the voxel location, the cavity volume can be classified asinternal cavity volume and external cavity volume. Internal cavityvolume is a portion of cavity volume which is surrounded by a part. In atubular structure, this internal cavity volume can represent the tubecavity volume. In contrast with internal cavity volume, external cavityvolume is a portion of cavity volume which is located outside of themodel boundary. The core contribution of the FICV process is how toidentify tube volume, which is called internal cavity volume amongcavity volume.

[0289] One of the main properties of an internal cavity is itsvisibility. In most cases, the internal cavity volume is hidden orblocked by the model so it is less visible than the external volume. Tomeasure how much visible a volume is, the degree of visibility isdefined. The degree of visibility identifies how much a volume isvisible. If the volume is completely visible in all directions, itsdegree of visibility is 1. If the volume is completely blocked by otherobject so that it is not visible in any direction, its degree ofvisibility is 0. In a case of the “A” cavity voxel (shown in FIG. 25),which is located on the entrance of tube volume, the voxel is halfvisible and half invisible and its visibility is 0.5. The basic strategyof FICV process is identifying the internal cavity volume using itsvisibility. First, FICV voxelizes cavity volume and measures a degree ofvisibility of each cavity voxel and determines a cavity voxel asinternal cavity if its degree of visibility is low.

[0290] To represent a degree of visibility in mathematical form, theconcept of solid angle is used with embodiments of this invention. Asolid angle is defined as the angle subtended at the center of a sphereby an area on its surface numerically equal to the square of the radius.FIG. 26 shows the concept of solid angle. Other than FIG. 26, thediagram might suggest, the shape of the area does not matter at all. Anyshape on the surface of the sphere that holds the same area will definea solid angle of the same size. Also, the diagram 2600 only shows theelements that define a solid angle, not the solid angle itself. Thesolid angle is the quantitative aspect of the conical slice of space,which has the center of the sphere as its peak, the area on the surfaceof the sphere as one of its spherical cross sections, and extends toinfinity. The maximum solid angle is about 12.57, corresponding to thefull area of the unit sphere, which is 4π. Standard unit of a solidangle is the Steradian (sr). (Mathematically, the solid angle isunitless, but for practical reasons, the Steradian is assigned.) If allthe visible directions are projected into the unit sphere, a degree ofvisibility can be defined as below:${{Degree}\quad {of}\quad {visbility}} = \frac{{visible}\quad {area}\quad {in}\quad {the}\quad {unit}\quad {sphere}}{{surface}\quad {area}\quad {of}\quad {unit}\quad {sphere}}$

[0291] Directional visibility is defined as whether a voxel is visibleor not in certain direction. If it is possible to know all directionalvisibilities of a voxel, then the degree of visibility can becalculated. Since the number of directions from a voxel is infinite, thecalculation cost to compute all directional visibilities of a voxel isvery high. To save the cost of computation a discrete directionalvisibility is calculated. The idea of discrete directional visibility isdefining equally distributed finite number of visible directions in avoxel and mapping the total number of visible direction to a visiblearea. If N equally distributed directions are defined and M visibledirections exist, the degree of visibility can be shown to be M/N. Togather equally distributed directions, this invention uses a N×N×N sizecube called a mask. FIG. 27 shows a 3×3×3 mask. In FIG. 27, eachboundary voxel represents the possible visible directions from a voxel.For example, a directional mask voxel “A” represents the direction(−1,−1, 1) in Cartesian coordinate. If the voxel can be seen in thedirection “A”, then a mask voxel “A: is marked as visible. Otherwise, itis marked as invisible. In this case, each mask voxel is assumed tooccupy 4π/26(total unit sphere surface area/a number of all possibledirections) Steradian in the unit sphere. After acquiring all 26directional visibilities, a degree of visibility can be calculated frombelow equation defines as:${{Degree}\quad {of}\quad {visbility}} = {\frac{{visible}\quad {area}\quad {in}\quad {the}\quad {unit}\quad {sphere}}{{surface}\quad {area}\quad {of}\quad {unit}\quad {sphere}}\quad = \frac{a\quad {number}\quad {of}\quad {visible}\quad {mask}\quad {voxels}}{a\quad {total}\quad {number}\quad {of}\quad {mask}\quad {voxels}}}$

[0292] The idea can be extended to the N×N×N mask. For example, a 5×5×5mask has 53-33=98 directional mask voxels on the boundary of the maskand a N×N×N mask has N³−(N−2)³=6N²−12N+8 directional mask voxels.

[0293] A directional visibility of direction (a,b,c) can be calculatedusing ray tracing method. A ray(a,b,c) is shot from the center voxel. Ifit go through any model voxel, then a voxel is not visible in (a,b,c)direction. In the other words, if a ray goes through all cavity voxelsand finally hits a boundary, then a voxel is visible in the direction(a,b,c). To determine a directional visibility of (a,b,c) direction onthe N×N×N mask, it is required to know the all position of voxels whichare passed through by the ray(a,b,c) from the center of the mask. A 3DBresenham algorithm is one of most popular algorithm to solve thisproblem.

[0294] With embodiments of this invention, it is demonstrated how toapply a 2D Bresenham algorithm for passing through voxels and extendingit to 3D space. FIG. 28 shows how the Bresenham algorithm works in thetwo dimensional space. Assume that u_(y)=v_(x) line in 2D plane in andthe pen has been stepped from the origin to the position labeled “P”.There can be two possible next steps: A diagonal step to point A and ahorizontal step to point B. If {overscore (AE)}<={overscore (BE)}, thenthe point A is closer to the exact line than the point B and otherwise,the point B is closer than the point A. Then, the algorithm selects thenext step as the closer point and repeat to find next step. Thealgorithm can be extended to 3D domain. Assume that a line$\frac{x}{u} = {\frac{y}{v} = \frac{z}{w}}$

[0295] in 3D plane and |u|>max(|v|,|w|). Then, it is possible to applyBresenham algorithm to P:(x,y) and Q:(x,z) domain and the result is aset of points as (x₁,y₁), (x₂,y₂), (x₃,y₃) . . . and (x₁,z₁), (x₂,z₂),(x₃,z₃) . . . . Then merge P domain and Q domain together by x. As aresult a set of points (x₁, y₁, z₁), (x₂, y₂, z₂), (x₃, y₃, z₃) . . .can be obtained.

[0296]FIG. 29 shows how the directional visibility can be gained usingBresenham algorithm in the 7×5 mask in 2D. There is a voxel A(0,0) anddirection mask voxel B(3,−2). To check Voxel “A”'s visibility ofdirection (3,−2), a process “FICV” checks all the voxels in the ray A-Bwhether the ray A-B is blocked by filled or model voxel or not. Afterapplying the Bresenham algorithm, all the voxel “c” and “d” {c(1,−1),d(2,−1), B(3,−2)} which are located between voxel “A” and directionalmask voxel “B” can be calculated. Each voxel is checked whether thevoxel is filled or not and the direction AB(3,−2) is considered asinvisible direction if voxel “c”, “d” or “B” is a model voxel. If not,the test is extended to next pattern by adding stepping vector (3, −2)to the all voxels. c(1,−1)->c′(4,−3), d(2,−1)->d′(5,−3),B(3,−2)->B′(6,−4). Then, check again whether c′, d′ and B′ voxels are amodel voxel or not. The test is extended until the range of the voxel isout of boundary. For saving a cost of calculation, 3D Bresenhamalgorithm variables are stored in each direction mask so that it canavoid a redundancy of calculation for every cavity voxels.

[0297] Once all the directional visibility information is calculated, itis possible to determine the voxel is internal cavity or external cavityby a degree of visibility which is a number of all visible directions inN×N×N mask divided by a total number of all possible directions in N×N×Nmask. According to statistical analysis, it is feasible to determine thevoxel as internal cavity if a degree of visibility is less than 0.30.

[0298] The result of identifying internal cavity volume can have manyapplications. One of the applications is detecting core and split corecandidates in casting area. In this invention, each internal cavityvolume represents a core candidate depending on a parting direction.After parting direction is configured, this invention can also beapplied to finding a number of cores and figure out a shape complexityof cores.

[0299] 2.2.4.3. Simplified Wireframe

[0300] The simplified wireframe skeleton starts with a voxelized modelthat has references to faces and edges of the 3D model. The objective ofthe simplified wireframe is to retain all ‘major’ features/portions of a3D model, while preserving the ‘major’ topology of the 3D model.Furthermore, a level-of-detail simplification can be performed. A voxelwireframe model is constructed by retaining all voxels that have two ormore faceIDs stored in the voxel as shown in FIG. 31.

[0301] The first step in obtaining a simplified wireframe is topreprocess the data and identify face loops. All face stacks areconsidered and they are sorted in order of descending number of voxelsin them. Pick the stacks in order of descending number of voxels. Foreach stack, start at a voxel and check adjacencies with the neighboringvoxels and form a linked list type of data structure with ‘previous’ and‘next’ pointers. The loop ends when the first voxel is arrived to again.If all the voxels in a stack have not been traversed then the face hasinternal loops. Start with all the voxels that have not been traversedand repeat the previous step of forming a linked list type datastructure. In this way one is able to identify all loops (connected ordisconnected) in a face. The remaining task is to identify which loopsare internal or external. Different methods can be used, but using anapproach to check the number of voxels for each loop in a face isbeneficial and optimal. The one with higher number of voxels is the‘External’ loop and all others are contained inside it, i.e. ‘Internal’loop. Thus all loops in a face stack are acquired with the distinctionof each loop being internal or external. It is now easy to see that theloops identified fall along the edges of the 3D model.

[0302] The next step is to traverse the loops in the voxel wireframe asshown in FIG. 30. Since we have all voxels and loops belonging to alledges in the 3D model, we can traverse the model along edges. Pick thelongest edge in the model i.e. the one having most number of voxels.Construct an N×N×N mask, N=2n+1 (i=0, 1, 2 . . . , n) depending on the3D model dimensions and other heuristics. Any voxel that falls withinthe mask while traversing the edges of the 3D model is marked as ‘D’(for Deleted). After having completed traversing a loop, check ifdeletion of voxels will change topology drastically. Topology changesare identified by direction vectors changing direction rapidly, asillustrated in FIG. 32. Connect the voxels that when deleted will changetopology drastically to the voxels that deleted them while traversing.While deleting voxels, the faceIDs and edgeIDs of deleted voxels arepassed on to the voxels that deleted them. This process is repeateduntil all loops are exhausted. This gives the first level-of-detail(LOD) simplification as shown in FIG. 33.

[0303] For future LOD simplifications, the number of voxels in eachretained loop is counted and the relation between loops is identified.i.e. are the loops intersecting, or do they share a common edge. Thiscan be identified from the edgeIDs and faceIDs stored in the voxels. Anexample of LOD simplifications possible for a model is shown in FIG. 34.

[0304] If CAD system manufacturers store the model feature history inthe CAD model and are able to roll back the CAD model, they are able tohave a LOD reduction like ours. One can use such an LOD reduction as askeleton.

[0305] 2.2.5. Levels of Detail

[0306] Different details in engineering parts can be captured by theskeletal graphs at different levels for resolution for the same part. Invarious embodiments, a 3D model is represented at different levels ofresolution by a hierarchical set of skeletal graphs. The varioushierarchical skeletal graphs are generated by varying the voxelresolution for the 3D model as described in. The number of voxels for a3D model is determined based on the smallest dimension of its boundingbox. The voxel size is calculated as s/2^(n) where s is the smallestbounding box dimension and n is the level of resolution desired. FIG. 35shows example pseudo code used for producing some skeletons generated byvarying the value of n.

[0307] Hierarchical structures are well supported by studies in humancognition. One such study suggests that that the perceptual approach toshape organization is dynamic. A partial order is apparent that relieson a hypothetical evolution or morphogenesis that is an integral part ofthe shape description. In the conventional approach, shapes that arevisually not all that different ultimately end up in entirely differentball parks. In general, human comprehension of an object's shape followsthe principle of “from remote to closer”, “from outer to inner”, “fromtotal to detail,”

[0308] The hierarchical skeletal graph structure is an example of adynamic approach to shape description. The advantage of using a dynamicapproach is that similar shapes can be detected at differentresolutions. For example, consider the shapes in FIG. 36. These shapesalthough visually similar, have very different shape descriptors throughconventional approaches. The skeletons for these shapes at the samevoxel resolution will also be different. However, they will yieldsimilar skeletons at individually different voxel resolutions. Thus, onecan detect them as similar at some level of detail.

[0309] 2.3. Graph Creation

[0310] The skeleton obtained by the above method is processed toidentify the high level entities—edges, loops, nodes. Edges are the setof voxels that form a basic geometric entity. Loops are a formed by asingle or group of edges constituting a closed path. Nodes are thevoxels that mark the ends/intersection of the edges/loops. An example ofskeletal graph terminology is shown in FIG. 37. Example skeleton edgeswith volume distribution are shown in FIG. 38. Moreover, an example CADmodel with example skeletons is shown in FIG. 39.

[0311] With various embodiments of this invention, a skeleton marchingalgorithm to identify these entities from the skeleton is depicted asexample pseudo code 4000 of FIG. 40. This marching algorithm 4000 shownin FIG. 40 uses a set of masks to identify the types of neighboringvoxels for each voxel in the skeleton. The algorithm 4000 selects astarting voxel for marching. An ordinary voxel has 2 neighbors. Thenumber of neighbors for each voxel is calculated as they are visited.When a voxel has more than 2 neighbors, it is a potential nodecandidate. Also when a voxel has only one neighbor in its 26 voxelneighborhood it is a terminal node. Further processing is done at theregion near these voxels to identify the correct nodes. The set ofvoxels visited between two nodes form an edge. This set of voxelsforming an edge are then assigned an entity id and stored in the entitystack. A curve fitting subroutine is used to approximate the geometry ofthe edges. This can be used for exact matching in the search process.

[0312] When the marching algorithm 4000 revisits any node, it means thata loop has been traversed. The entity stack is then processed toidentify the edge or set of edges which form the loop. The algorithm4000 maintains a node stack and processes the branches at each node oneby one. Once all the voxels are visited and the related edge/loopprocessing is done, the algorithm 4000 is complete. In the case ofmultiple loops in the same skeleton, some of the loops share one or manyedges with other loops. The loops which do not share any edge or nodewith the other loops are called simple loops. The marching algorithm4000 identifies all the simple loops. To get all the non-simple loops inthe skeleton, some post processing needs to be done. The skeletal graphis analyzed based on the depth first technique to identify the multipleloops sharing an edge or a node.

[0313] Edge segmentation deals with converting the edges identifiedbetween junction vertices to simple geometric entities that can beeasily compared. This involves checking whether a set of voxels belongto a straight line or planar curves without torsion. If the voxels donot form a planar curve, the curve is projected onto two perpendicularplanes and the torsion values are used to segment the curve into planarcurves. Each of the voxels in a set is treated as a discreet point in3-space. A line is initially constructed between two newly found endpoints of the set of voxels. The distances between such straight linesand the farthest voxels in the set are used to determine new verticesand to form new simpler entities. This in turn causes new nodes andedges to be created in the skeletal graph. This process outputs simpleentities that can be easily compared in 2D using curvatures at thelow-level graph-matcher. FIG. 41 shows the subsequent steps in detectingnew vertices and converting the curve to a set of simple segments (twoplanar curves C₁ and C₂, and a straight edge E₃). The diagram on theright of FIG. 41 shows the consequent changes in the graph structure dueto creation of new entities.

[0314] Finally, an edge-loop list of the skeletal graph is constructedwith pointers to the skeletal nodes. This graph is stored in thedatabase for future search. Thus the geometry of the 3D model iscaptured in the individual entities of the skeleton (FIGS. 38 and 39)and the topology in the skeletal graph.

[0315] 2.4. Feature Vectors Extraction

[0316] The skeletal graph is further processed to identify the graphsrelated feature vector components which can be used in the search. Someof the main components are the number of edges, loops, nodes etc. Theskeletal graph is further processed to identify the degree oftopological connections between the entities at the nodes. The differenttypes of topological connections are represented by suitable Laplaciancoefficients in the entity adjacency matrix calculated from the skeletalgraph. More attributes are derived from the skeletal graph for use inthe graph comparisons as described below.

[0317] 2.5. Conversion of 2D Drawings into 3D Skeletons

[0318] The primary challenges of manufacturing companies are to designbetter products, in shorter time frames and at lower cost. Recentlythere has been a remarkable shift from 2D CAD design techniques totechniques that take the advantage of 3D solid modeling. With lowercosts and universal accessibility to computer-aided tools, nearly allmanufacturing firms have some kind of CAD systems installed. A 1998survey by Mechanical Engineering magazine indicates that 96% ofmechanical engineering professionals currently use a CAD system. ButDespite the advent of affordable and easy to use 3D modeling technology,a majority of manufacturing firms still base their design processes on2D CAD techniques and drawing data. A 1998 survey by the computer aidedEngineering magazine revealed that more than 60 percent of CADengineering work is done in 2D. Thus, there is a need for a searchmechanism, which will allow designers to search for similar 3D modelsand 2D drawings for a given 2D drawing. Such a search system will enablecompanies to reuse their legacy data to the optimum. Most of therelevant work in the field focuses on the content/context based searchmechanisms for retrieving similar parts and images. Geometry basedsearch mechanisms have also been proposed to retrieve similar/matching2D images & 2D drawings from a database. One such system has beenproposed where images (drawings) are represented using attributed graphsbased on extracted line-patterns or histograms of attributes computedfrom the graphs. Retrieval is either performed using histogramcomparison or graph matching. Although, the method adopted by thisapproach includes the shape information, the fact that the threeorthographic views are related to each other is not utilized to thefullest extent. These techniques compare different views to determine iftwo drawings match or not. Embodiments of this invention use an approachto convert the 2D drawing to a 3D model and search for similar models in3D. The 3D representation obtained can be used to search for matching 3Dmodels, thus allowing the search for 2D drawings and 3D models from adatabase of CAD drawings/models. This permits reuse of legacy data. A 3Dshape search application has already been presented, which can use thesereconstructed 3D models to search for similar models. The 3D model isfirst converted to a 3D skeleton and then it can be used for searching.Therefore, the technique is to come up with a 3D skeleton, which can beused by the 3D shape search engine.

[0319] Construction of solid models from engineering drawings continuesto be a problem of interest. The work in this area dates back to nearlythree decades. In these three decades many different approaches havebeen adopted to solve the problem, with fair amount of success. But thefact that the problem still remains a topic of interest is an indicationof the complexity of the problem. The major driving factors in thisfield are design reuse, automatic verification of the consistency ofcomplicated drawings and wide applicability. These generated solidmodels find applications in FEA, CFD, or simulation to analyze, optimizeor evaluate the design and in manufacturing for rapid prototyping and NCpath planning. There are two broad approaches for converting drawings tosolid models. The existing methods of reconstruction of solid modelsfrom 2D drawings can be broadly categorized into two types:

[0320] Wire-Frame oriented approach or bottom-up approach.

[0321] Volume oriented approach or to-down approach.

[0322] The bottom-up approach assembles candidate vertices, edges andfaces, in that order to construct the solid model. The top-down approachcreates elementary solids by recognizing patterns in 2D projections andassembles them to get the solution.

[0323] Thus, embodiments of this invention convert a 2D drawing to a 3Dskeleton.

[0324] Methodologies:

[0325] Conversion of a 2D drawing to a 3D skeleton can be achieved inmany different ways. Some of the methods are listed as below:

[0326] a) Wire-Frame based Approach: Legacy data mostly exist as scanneddrawings. Hence, it is most likely that input to can be a scanneddrawing. The discussion hereafter will assume that the input is ascanned drawing. The wire-frame based approach will follow the followingprocessing:

[0327] Pre-processing: This step converts the scanned 2D drawing into avectorial format. The vectorized output contains data pertaining to thethree orthographic views.

[0328] Vectorized drawing to 3D wire-frame: The three orthographic viewsare combined together to get a 3D wire-frame model.

[0329] 3D wire-frame to 3D model: The 3D wire-frame model is thenconverted into volume based 3D model.

[0330] 3D Model to 3D skeleton: The 3D model is then normalized andskeletonized to obtain a 3D skeleton. The processing after obtaining the3D skeleton has been described in section 1.2.

[0331] b) Volume-Based Approach: The volume based approach will followthe following processing:

[0332] Pre-processing: This step converts the scanned 2D drawing into avectorial format. The vectorized output contains data pertaining to thethree orthographic views.

[0333] Vectorized views to 3D model: The three orthographic views modelis then converted into volume based 3D model.

[0334] 3D Model to 3D skeleton: The 3D model is then normalized andskeletonized to obtain a 3D skeleton. The processing after obtaining the3D skeleton has been described in section 1.2.

[0335] c) Direct conversion to 3D Skeleton without obtaining anyintermediate 3D model:

[0336] Pre-processing: This step converts the scanned 2D drawing into avectorial format. The vectorized output contains data pertaining to thethree orthographic views.

[0337] The Vectorized orthographic views are directly converted into a3D skeleton: This process has been explained in section 2.3 above.

[0338] It is also important to note that immediately after thepre-processing is completed, normalization is done, to eliminate anysize related errors.

[0339] Terminologies Used:

[0340] Terminology used is described below for purposes of readilycomprehending various embodiments presented herein.

[0341] 1) Common Coordinate Axis: It is the coordinate axis that iscommon between pair of views.

[0342] 2) Fundamental Edge & View-Edge: A 2D edge GE in a view G is saidto be fundamental-edge, if there does not exist any vertex in view G,which lies IN edge GE Otherwise, it is called as view-edge.

[0343] 3) Loop:_A simple closed cycle of coplanar edges is defined as aloop.

[0344] 4) Tangency vertex: A vertex in a 2D view that separates a pairof edges, which are first order continuous, is said to be tangencyvertex. Since the notion of tangency vertex is associated with a pair ofedges, it is used for vertices in a loop.

[0345] 5) Silhouette vertex: A tangency vertex ^(G1)V in a 2D view G1 issaid to be silhouette with respect to an adjacent view G2, if thetangent to the edges at that vertex is perpendicular to the commoncoordinate axis between G1 and G2. Silhouette vertex is thereforedefined for a pair of views.

[0346] 6) Classification of loops: A loop ^(G)L₂ can be classified withrespect to the loop ^(G)L₁ as follows:

[0347] Loop ^(G)L₁ is said to have IN loop ^(G)L₂, if all edges ^(G)Ebelonging to ^(G)L₂ lie inside loop ^(G)L₁.

[0348] If some edges belonging to ^(G)L₂ lie inside and others lie onthe boundary of the region P, then ^(G)L₁ is said to have ON-IN loop^(G)L₂.

[0349]^(G)L₁ is said to have a ON-OUT loop ^(G)L₂, if some of the edgesbelonging to ^(G)L₂ lies outside and others lie on the boundary of theregion P.

[0350] 7) Dotted Loop & Solid Loop:_A loop in 2D view is said to be asolid loop, if all of its edges are represented as solid line type inthe view. Otherwise, it is called as a dotted-loop.

[0351] 8) Adjacent loop: If two loops in a view have one or more commonedges, then these loops are said to be adjacent loops.

[0352] 9) Fundamental Loops and Non-Fundamental loop: A Fundamental loopis an edge loop that does not have any ON-IN loop in the view but it mayhave an IN loop. All other loops are called non-fundamental loops. FIG.42 presents example classifications of loops. In FIG. 42{1,2,3,9,8,7,5,6,1}, {8,9,10,7,8} and {3,4,5,7,10,9,3} in view 1, arefundamental loops.

[0353] 10) Disjoint Loop: A disjoint loop is that fundamental loop whichdoes not have any ON-OUT loops in the view but may have an OUT loop.

[0354] 11) Matching Vertices: A vertex ^(G1)V in a view is said to bematching with a vertex ^(G2)V in another view G2, if they have the samecoordinate value along the common coordinate axis. The matching verticescan be denoted by ^(G1)V<-->^(G2)V. There may be more than one matchingvertices in a view matching a vertex in other view.

[0355] 12) Matching Loops: A loop ^(G1)L belonging to G1 is said to havematching loop G2L belonging to G2 (G1!=G2), if every vertex^(G2)V′∪^(G2)L matches with at least one vertex in ^(G1)V∪^(G1)L.^(G2)V′ is obtained by removing all tangency vertices that are notsilhouette vertices for the pair of views. Note that it is not necessarythat ^(G2)L also have ^(G1)L as a matching loop. But if it does, thenthe two loops are known as onto-matching loops. They are denoted as^(G1)L<-->^(G2)L.

[0356] 13) Onto formation: An onto-formation is a set of loops(l)satisfying following properties:

[0357] a. For i,j=1 , . . . n (i!=j), ^(Gi)Li<-->^(Gi)Lj, where ^(Gi)Li,^(G)Lj belong to set 1.

[0358] b. |l|=n, where n is number of input 2D views.

[0359] 14) Desirable Edge: It is an edge in the input orthographic viewsthat is not there in the 2D projections of the current solid.

[0360] Example Embodiment of Method:

[0361] In the following sections, the processing for the abovereferenced methodologies is described:

[0362] 1. Preprocessing:

[0363] In order to solve the problem of 2D shape searching, one firstneeds to transfer the document to electronic formats. The most widelyused method is scanning, which produces “raster images”. The scanneddrawing is broken down into a matrix of pixels. But the semanticinformation contained in the document is not transferred to electronicfile, i.e. the meaning of the symbol is not interpreted. Hence, somepreprocessing steps known, as “Vectorization” is needed, by means ofwhich the information necessary to define geometrical entities (such asstraight lines, arcs, etc.) and symbols (such as letters) is obtainedfrom a raster image. Hence the information stored on an electronic filecan be interpreted and used for searching based on feature and reusedbased on solid model/skeleton reconstruction.

[0364] Scanning Type

[0365] Most engineering drawings can be scanned as 1-bit monochrome. Ittakes less storage space and is faster in displaying and processing. Fordrawings with dirty and smearing background, such as old blue prints,they can be scanned as 8-bit grayscale and enhanced using imagingsoftware to remove background and noise (Noise and other artifacts canbe easily smoothed out using a pair of gray level thresholds beforeautomatic vectorization).

[0366] Raster to Vector Conversion

[0367] After obtaining a scanned raster image, one needs to convert itinto vector format for further manipulation. A complete raster to vectorconversion process includes image acquisition, pre-processing, linetracing, text extraction, shape recognition, topology creation andattribute assignment. Several raster to vector conversion softwarepackages are commercially available for various types of applications.For engineering drawing conversion, in order to extract vectors from araster image, it needs to determine which parts of the image constitutelines, and where those lines start and end. Typically there are twoprocesses involved:

[0368] 1. Treatment of the image (remove background and noise).

[0369] 2. Thinning of the raster image to single pixel width lines.

[0370] 3. Vectorization (Extraction of the edge from the pixel lines).

[0371] 4. Entity recognition (define straight lines, arcs, circles, lineloops, etc.)

[0372] The approach to reduce thick or blobby regions down to singlepixel width items, so that the image is transformed into lines of pixelsis called thinning. There are many current available thinning and edgedetection methods, each utilizing different mathematical algorithms andproducing different results.

[0373] All of these preprocessing steps involve image processing forunderstanding the information contained in the images in order toextract the relevant shape information.

[0374] 2. Conversion of 3D Orthographic Views into 3D Model UsingBottom-Up Approach:

[0375] The wire-frame approach is also known as bottom-up approach. Thevarious processing involved in this method are as follows:

[0376] Generate 3D candidate vertices from 2D vertices in each view.

[0377] Generate 3D candidate edges from 3D candidate vertices

[0378] Construct 3D candidate faces from 3D candidate edges on the samesurface

[0379] Construct 3D objects from candidate faces.

[0380] Merits of this Approach:

[0381] More accurate

[0382] Other metrics which are usually obtained from ACIS can also beobtained

[0383] De-Merits of the Approach:

[0384] Problems may arise as a result of inconsistency in the inputviews

[0385] It is computationally very expensive. Especially, the conversionfrom a wireframe to solid is very computationally expensive, as itinvolves search with possibility of backtracking and heuristics tochoose solution solid from a very large number of potential solids.

[0386] 3. Conversion of 3D Orthographic Views into 3D Model UsingTop-Down Approach:

[0387] The general processing, involved in this approach are describedbelow:

[0388] 1) Construct pseudo vertex skeletons.

[0389] 2) Construct pseudo wireframes.

[0390] 3) Construct virtual faces.

[0391] 4) Introduce cutting edges.

[0392] 5) Construct virtual blocks.

[0393] 6) Make decisions.

[0394]FIG. 43 presents more details of this process. There may be othermethods as well. In another embodiment, a cell-based reconstruction isused. Similarly, two-phase reconstruction has also been developed.

[0395] Merits of this Approach:

[0396] Always have a valid solid model consistent with the input views.

[0397] Compared to wire-frame method, computationally inexpensive.

[0398] For shape searching, only an approximate 3D model is sufficient.

[0399] De-Merits of the Approach:

[0400] This approach is limited to only domain of objects they canhandle.

[0401] 4. Direct Conversion to 3D Skeleton Without ObtainingIntermediate 3D Model:

[0402] As mentioned before, for 3D shape searching a valid 3D skeletonis sufficient. Hence, a method is presented for converting a given setof 2D orthographic views directly into a 3D skeleton. The 2D skeletonsare obtained for each of the three orthographic views; the orthographicviews are then combined to obtain the desired 3D skeleton. This methodcan be further subdivided into two approaches depending upon the 2Dskeletonization process. The first approach uses image-processingmethods to perform the skeletonization, and the second approach usesgeometry based methods to do it. The following discussion assumes thatgeometry based approach for obtaining the 2D skeleton, but the approachcan be modified suitably to work with the first approach as well.

[0403] Formation of Doubly Connected Linked List and Face Structure:

[0404] The line segments forming the three orthographic views are thenexpressed in a doubly connected linked list structure. Methods have beenproposed for building a doubly connected linked list structure. A doublyconnected linked list is an efficient representation of arrangement ofline segments.

[0405] The data structure for vertex is expressed as:

[0406] X-coordinate of the vertex.

[0407] Y-coordinate of the vertex.

[0408] List of Incident Edges (ordered in counter-clockwise direction ofthe slope).

[0409] The Data structure for the Half Edge is given as:

[0410] Origin vertex.

[0411] Twin Half-Edge (the one coming in)

[0412] Incident Face (every half edge belongs to exactly one face)

[0413] Next Half-Edge (the next connected half-edge belonging to theface).

[0414] Previous Half-Edge (the previous connected half-edge belonging tothe face).

[0415] Slope angle (between 0 to 360 degrees)

[0416] The data structure for a face is given as:

[0417] Starting edge of the outer loop.

[0418] List of starting edges of inner loops.

[0419] Hence, to travel the incident edges on a vertex incounterclockwise direction, one can use the above data structures tofull advantage. Given an edge incident on the vertex, one takes the twinof that half edge and finds the next half edge to that half edge. Thiswill give us an incident half edge from the given vertex. The obtainedhalf edge is the next in counter-clockwise direction, because thehalf-edges forming the faces have been obtained by walking incounter-clockwise direction through the edges. The same procedure can berepeated, to find all the faces/loops in the arrangement. The datastructure and details of the algorithm are conventionally available. Thearrangement of the set of line segments can be constructed in O(n Log n)time.

[0420] Identification of Matching Vertices, Edges and Loops of the ThreeOrthographic Views

[0421] The identification of matching vertices, edges and loops has beenconventionally explained in detail and is readily available to one ofordinary skill in the art. The method for finding matching loops andvertices has also been described:

[0422] 1. Matching Vertices and Loops: Matching of loops and verticesimplies grouping of the entities that correspond to the same entity inthe 3D solid. The tangency vertices other than silhouette vertices arenot used while matching a pair of two loops in different views. Andvertices are matched according to the value of common coordinate. Ifloops are matching each other, then they are flagged as onto-matching.

[0423] 2. Onto formation: For finding an onto-formation, loops matchinga disjoint loop or a fundamental loop are considered. A loop in view canhave fundamental or non-fundamental matching loops in the adjacentviews. First of all, all fundamental loops matching the loop underconsideration are checked, if they are onto-matching. Any onto-matchingloops obtained are stored in a list. Next, it is checked if it ispossible to obtain a non-fundamental onto matching loop by merging ofadjacent fundamental loops, each of which matches the fundamental loopin the other view under consideration. These are stored for furtherprocessing. The data structure used for storing these entities is agraph.

[0424] In general, onto-formations are identified across the given setof views. In FIG. 44, loops are represented as nodes and edge joiningtwo nodes shows that the two loops are onto-matching. This can be seenas a n-partite graph of loops, where n is the number of loops. Sofinding onto-formation is equivalent to finding a sub-graph ofcardinality n. A breadth-first-search is used to find this in n log(n),where n is the average number of loops under consideration. Anonto-matching set of loops is a set of loops in all the views, which areonto-matching with each other.

[0425] Cleaning Up the Three Orthographic Views:

[0426] After the set of onto-matching loops have been obtained, thoseset of onto-matching loops are eliminated, which describe smallfeatures. Care should be taken that the set of onto-matching loops areonly disjoint or fundamental loops. Also, the resulting views should notbe disconnected as a result of removal of these loops.

[0427] 2D Skeletonization:

[0428] In the next step, these loops or faces are skeletonized tosimplify the drawing while retaining the topology. Various methods havebeen proposed to obtain the skeleton for polygonal figures. Most ofthese methods determine the Medial Axis Transform (MAT). Most of thesemethods either make use of Voronoi diagrams or Delaunay Triangulations.There have been other approaches, which obtain the skeleton or MATwithout using Voronoi or Delaunay triangulation. Also, adaptivesubdivision algorithms have been proposed, which make use of Delaunaytriangulations. A skeleton of a 2D region is defined as a set of itsinterior points that have not less than two nearest neighbors. Theskeleton of polygonal figures can be obtained in O(n log n) worst casetime, where n is the number of nodes of the polygonal figure boundary.FIG. 45 shows the skeleton for a general 2D polygonal figure.

[0429] Cleaning Up the Skeleton:

[0430] It can be observed from FIG. 45 that some portions of theobtained 2D skeleton are not required. The edges of the skeleton, whichtouch the boundaries/vertices of the input views, are then eliminated.Hence, a simplified skeleton is obtained (shown in FIG. 46).

[0431] Using Matching-Loop and Adjacent-Loop Relations Connect theseEdges of the Skeleton in the Three Views to Obtain a 3D Skeleton:

[0432] Once the skeletons of various faces have been cleaned up, thenext step is to connect the skeletons in the three views to arrive atthe 3D skeleton. Referring now to FIG. 44; FIG. 44A shows an actual 3Dmodel; and FIG. 44B shows the 2D orthographic views for the givenobject. FIG. 44C shows the cleaned up skeleton for each of the views.Using the matching loops obtained in the first 2nd step, theconnectivity of various edges of the 2D skeletal edges is determined. Tomaintain the connectivity all matching loops are connected, and also itis made sure that each edge in the 3D skeleton appears just once. Forexample, in the front view, the vertical line denotes the depth/heightof the figure. Also in the side view the 5 vertical lines, too, give thedepth of the figure. Hence in the final skeleton, only one vertical edgeis retained to denote the depth of the figure.

[0433] Edges of 3D Skeleton are Voxelized:

[0434] Once the 3D edges are obtained, they are then converted intovoxelized representation. Basically, voxelized representation is adiscretized representation of the edges in the 3D skeleton. This step isa simple processing step. For more information on Voxelization andNormalization refer to subsection 2.1 above.

[0435] Store the Connectivity of Various Loops and Edges in aData-Structure:

[0436] Finally, when the 3D skeleton is obtained, the various loops,edges, etc. are stored in a skeletal graph data structure as describedin subsection 2.2. This is then provided as input to the graphcomparison module.

[0437] 3. Database, Search and Clustering

[0438] 3.1. Compare Graphs

[0439] The search system searches for graphs in the database which aresimilar to the skeletal graph of the query 3D model. The systemevaluates a similarity measure based on the degree of matching betweenthe query graph and each of the models in the database. The currentinvention represents the skeleton in the form of an undirected graph,where a node of the graph represents an edge/entity of the skeleton,while any arc joining two nodes of the graph represents the connectionbetween the corresponding skeletal entities. Hence, the skeleton isessentially represented as an edge-graph. Each node in the graphpreserves the following geometric properties of the skeletal edge:

[0440] (1) Edge type (straight line, curve, surface loop or hole).

[0441] (2) Curvature information for surface loops(Convex/Concave/Plane).

[0442] (3) Parametric equation of the curve.

[0443] (4) Local Volume of the features that converge to the particularentity.

[0444] (5) Local Moments of the features that converge to the particularentity.

[0445] (6) Local Distances from the surface.

[0446] Hence, the topology of the skeleton is captured by the topologyof the graph, while the geometry is preserved in the nodes of the graph.FIG. 47 presents a flow chart of the graph matching module of thecurrent system.

[0447] High-Level Graph Matching

[0448] The first two properties (1 and 2) cited above are used for ahigh-level graph matching step (see FIG. 47) after which a small subsetof the database having similar models is retrieved. These models haveoverall geometric and topological similarity with the query model. In afollow-up step the retrieved models are further ranked (low-level graphmatcher), based on the similarity of the individual geometric properties(3), (4) and (5) as described above. Hence, the high-level matcherfilters the search space to find relevant parts. This filter uses graphproperties including the highest degree of nodes, number of surfaceloops, number of straight and curved edges, and the number of holes inthe skeleton to filter out graphs that are not close to the query model(or query graph).

[0449] The smaller set of models retrieved by the high-level graphmatcher is then passed onto the low-level graph matcher. The low-levelgraph matcher which finds a similarity degree between the query modeland these models, based on a graph matching algorithm as describedbelow. However, the user may also choose to perform matching only at ahigher level of abstraction using topology and high-level geometricproperties. To enable this matching, the adjacency matrix of the graphis designed to represent these geometric properties. A non-zero entry inthe matrix represents a connection between two geometric entities(topology). The value of the entry uniquely represents the type ofconnection (edge-loop, loop-loop, surface-surface, edge-edge, and soon). This adjacency matrix can be used for any graph matching methodincluding the decision-theoretic sub-graph isomorphism algorithm as wellas association graph techniques.

[0450] Low-Level Graph Matching

[0451] Various algorithms have been proposed in the past for comparinggraphs and evaluating a similarity measure as described in the prior artsection. These algorithms can be broadly classified into exact andinexact graph matching algorithms. While the former class of algorithmsdetects an exact match (i.e. without noise) between two graphs orsub-graphs of two graphs, the latter set of algorithms provides a fuzzymeasure of similarity between two graphs in the form of a distancemeasure even in the presence of noise. Since the requirement of oursystem is to find parts that are similar (not exactly same), our systemuses inexact algorithms for the purpose of graph comparison.

[0452] Traditional graph isomorphism for graph comparison leads toNP-complete problem, i.e. it is not possible to find a solution using acomputer except for small graphs. The field of inexact graph matching(also called error-tolerant graph matching) has been studied for manyyears leading to many different frameworks of solutions. The associationgraph technique has been applied successfully to compare graphs invarious applications including similarity of scene graphs. In thecurrent system, the graph matching problem is converted into a maximumweighted-clique problem by forming the association graph for the queryand model graphs. In the current invention probabilistic weights areincluded in the association graph nodes. These weights are evaluatedbased on the local geometric feature similarity of the entitiesincluding (1) curvatures, (2) local volumes, and (3) local moments. Thisallows for fuzziness in the similarity measure between models so thatthe relevant models are not rejected.

[0453] Association Graph Technique

[0454] Although, the association graph method is well-known in the fieldof graph matching for computer vision and pattern recognition, themethod is describe here (in the context of our problem) for the sake ofcompleteness along with the modifications to make it suitable for ourproblem. The nodes of an association graph (or assignment graph)represent the vertex pairs (v_(Q), v_(M)) of the graphs to be compared(say, G_(Q)=(V_(Q), E_(Q)) and G_(M)=(V_(M), E_(M))). A vertex pair(v_(Q), v_(M)), such that v_(Q)∈V_(Q), v_(M)∈V_(M) is called anassignment if the nodes v_(Q) and v_(M) have the same node propertydescriptions, and two assignments (v_(Q), v_(M)) and (v_(Q)′, v_(M)′)are compatible if (in addition) all relations between v_(Q) and v_(Q)′also hold for v_(M) and v_(M)′. The set of assignments defines the setof nodes V_(a) of the assignment graph G_(a). Two nodes, V_(a) andV_(a)′, are connected by an arc in the assignment graph, G_(a), (seeFIG. 48) if these two nodes are compatible. Hence, the search formaximum matching sub-graph of two graphs G_(Q) and G_(M) is a search forthe maximum clique (totally connected sub-graph) of the associationgraph G_(a). Instead, if the assignments also represent a measure ofsimilarity between v_(Q) and v_(M), represented as weights, then thegraph matching problem is converted into a maximal weighted-cliqueproblem. Hence, the largest maximal weighted-clique in the associationgraph will represent the best match between the two graphs G_(Q) andG_(M).

[0455] Local Similarity Metrics

[0456] This section is devoted to describe a shape matching algorithm indetail which is based on an association graph technique. Although thewords ‘‘node’ and ‘vertex’ as well as ‘edge’ and ‘arc’ areinterchangeably used in graph theory, for the sake of clarity a skeletalgraph is described as using nodes and edges, while an association graphis made up of vertices and arcs. Vertices of an association graphrepresent the correspondences between nodes of the two original graphs.This process of determining correspondences between nodes is calledassociation. In addition the arcs connecting two vertices of theassociation graph represent the compatibility between the correspondingnodes.

[0457] A graph A is represented as set of vertices A_(i), a set of edgesAij, and set of vertex attributes, v(A_(i)) and edge attributes.

[0458] A skeletal graph structure, in various embodiments presentedherein, distinguishes between edges and loops; associations are allowedonly between entities of the same type, and thereby associates loops ofone graph only with loops of the other skeletons and so on. Similaritybetween associated nodes (say A1 and B2) is determined based on thelength similarity as well as degree similarity. Weights are assigned tothese two metrics.

[0459] Hence, the similarity between node A1 of graph A and B2 of graphB is represented as the weight of the nodes and is given by Equation(1). $\begin{matrix}{{{VertexSimilarity}\left( {A_{1},B_{2}} \right)} = {{W_{l}\left( {1 - {{l_{A\quad 1} - l_{B\quad 2}}}} \right)} + \quad {W_{d}\left( {1 - \frac{{d_{A\quad 1} - d_{B\quad 2}}}{\max \left( {d_{A\quad 1},d_{B\quad 2}} \right)}} \right)}}} & (1)\end{matrix}$

[0460] Here, I_(E) _(i) is the length of entity E_(i) while d_(E) _(i)is the degree of entity E_(i). W₁ is the weight given to the lengthsimilarity, while W_(d) is the weight given to similarity between theskeletal entities based on their degree (number of other entitiesconnected to the entity under consideration). A higher weight on thedegree of nodes will retrieve models more similar in topology while ahigher weight on the length similarity will result in models with highergeometric similarity. Correspondingly, the user can also change theseweights depending on the search criteria. Additional metrics comparingother local properties including (i) Volume Distribution, (ii) DistanceTransform, (iii) Local Principal Moments, and (iv) Local Curvature.These terms are added to equation (1)

[0461] Similarly, the arc of an association graph represents whether thecorresponding entities are connected and if so the relative similaritybetween the entities. For example, the arc connecting the nodes (A1-B2)and (A2-B3) will describe whether A1 is connected to A2 and whether B2is connected to B3. In addition, it may also contain other heuristicsthat better describe the connections between the corresponding entities,such as the angle between connected entities. In one implementation themeasure given by Equation (2) 1 used to determine the weight on the arc.$\begin{matrix}{{{EdgeSimilarity}\left( {{A_{1}A_{2}},{B_{2}B_{3}}} \right)} = {{W_{lr}\left( {1 - {{\frac{l_{A\quad 1}}{l_{A\quad 2}} - \frac{l_{B\quad 2}}{l_{B\quad 3}}}}} \right)} + \quad {W_{\theta}\left( {1 - \frac{{\theta_{A\quad 1A\quad 2} - \theta_{B\quad 2B\quad 3}}}{\pi}} \right)}}} & (2)\end{matrix}$

[0462] Here, θ_(E) _(i) _(E) _(j) is the angle between entities E_(i)and E_(j).

[0463] W_(tr) is the weight given to the ratio of lengths while W_(θ) isthe weight given to angles. It is noted that a number of constraintshave to be taken care of, in order to obtain valid graph matching. Forinstance, a given skeletal entity of model A cannot be matched to morethan one entity of model B, and vice versa. These constraints are shownin FIG. 48 as ovals. Arcs of the association graph are formed with theseconstraints in consideration.

[0464] Subsequently, the graph matching problem between model A andmodel B is now posed as a maximum clique problem. Due to the presence ofweights in the association graph in the application, the problem isformulated as a maximum-weighted clique problem. The clique with thelargest weight, therefore, represents the best similarity between thetwo graphs. Similarity is therefore the sum of the node weights and thearc weights for a given clique. Hence, the problem of finding the ‘bestsimilarity matching’ between two models is essentially a combinatorialoptimization problem where the objective function is the similaritymeasure.

[0465] A heuristic based Genetic Algorithm is used to exploit themaximal weighted clique of the association graph described above;however, any combinatorial optimization algorithm can be employed forthe obtaining the maximal weighted clique. Although approximatealgorithms do no guarantee globally optimal solutions for large graphs,they avoid exhaustive search which becomes intractable for large graphsizes. Hence, approximate algorithms are advantageous in variousembodiments for optimization while using the approach.

[0466] Computational Complexity

[0467] The main advantage of using the association graph technique isthat it is a simple graph-theoretic structure which is amenable to puregraph-theoretic algorithms such as maximum clique finding. However,construction of the association graph is purely domain dependent.Therefore, the effectiveness of this approach depends on theconstruction of this graph and subsequent utilization of thegraph-theoretic algorithms. It is noted that the maximum cliquedetection problem is NP-complete. However, the presence of richgeometric information in our skeleton graph will enable us to reduce theaverage-case complexity of the maximum clique finding problem (MCproblem) considerably by eliminating many unnecessary comparisons. Thealgorithm is designed to utilize heuristics including a) lexicographicordering of nodes b) size of the previous cliques and c) number ofhigh-level geometric entities (Surface Loops/Holes/Edges) to prune thesearch space. For example, one can eliminate comparisons of loops withstraight edges and curves, or curves with straight edges. Loops in theskeletons essentially represent major holes in the original 3D model.Hence, some of these features implicitly represent the manufacturingfeatures in the original 3D models. This is an illustration of how theuse of domain knowledge for graph matching can considerably reduce theaverage-case complexity. The modifications to the association graphtechnique permits one to achieve searches on 3D models in a reasonabletime. This technique is better illustrated in the following discussion.

[0468] Suppose a query graph consists of Q nodes out of which Q₁ nodesrepresent straight edges, while Q₂ nodes represent loops. Similarly, letM₁ and M₂ be the number of nodes representing straight edges and loops,respectively, in the model graph (with a total of M nodes). Let us alsoassume that M₁, M₂ and M are individually greater than Q₁, Q₂ and Q.Hence, we find that the number of comparisons required for sub-graphisomorphism as: $\begin{matrix}{{{Number}\quad {of}\quad {{Comparisons}\left( {w\text{/}\quad {attr}} \right)}} = {\frac{M_{1}!}{\left( {M_{1} - Q_{1}} \right)!} \times \frac{M_{2}!}{\left( {M_{2} - Q_{2}} \right)!}}} & (3)\end{matrix}$

[0469] However, if the node attributes (Surface/Loop/Edge) are not usedthen the number of comparisons is given by following expression:$\begin{matrix}{{{Number}\quad {of}\quad {{Comparisons}\left( {w\text{/}o\quad {{attr}.}} \right)}} = \frac{M!}{\left( {M - Q} \right)!}} & (4)\end{matrix}$

[0470] It may be observed that a large number of unnecessary (naïve)comparisons are eliminated using the geometric information present inthe graph structure thereby reducing the computational complexity ofmatching. Moreover, the number of nodes which need to be formed in theassociation graph without this information would be M×Q. However, withthe approach we reduce this number to (M₁×Q₁)+(M₂×Q₂). Theaforementioned simplifications reduce the size of our association graphsin the average case, although the worst-case complexity is exponentialin the number of nodes in the input graphs. The worst-case occurs when askeleton is made of the same type of entities (i.e. all loops, allstraight edges, etc.). The graph matching algorithms described above canalso be extended for shell-like and hybrid parts. Geometric featuresincluding 1) convexity 2) radius of curvature of the surface loops canbe further used to reduce the search space for these parts.

[0471] Geometric Feature Matching

[0472] Geometric feature attributes of the skeletal entities arecompared in order to provide more accurate measures ofsimilarity/distance between the models retrieved from high levelmatching. As mentioned above, dissimilar entity types will not becompared for graph similarity metric.

[0473] Prismatic Shapes

[0474] Bezier curves are used to obtain a parametric equation of theprismatic skeletal entities. This provides an affine invariant measureof the curve shape. An algorithm based on the curvature for curvesimilarity comparison is used and similar curves have similar curvatureprofiles. To compare two curves the curves are divided into a definednumber of entities and the curvature profile for the curve is thesequence of these curvatures from one end of the curve to the other. Afuzzy measure of similarity is derived from the distance between thesetwo curvature profiles. Equations for the Bezier curve and the curvatureat any given point on the curve are provided below.

[0475] Fifth-order Bezier curve fitting${p(t)} = {{\sum\limits_{0}^{n}{\begin{pmatrix}n \\i\end{pmatrix}\left( {1 - t} \right)^{n - i}t^{i}p_{i,}0}} \leq t \leq 1}$

[0476] n=5

[0477] p_(i)—Control Points

[0478] (_(i) ^(n))(1−t)^(n−i)t^(i)—Bernstein Polynomials

[0479] Curvature from equation${{Curvature}\quad \kappa} = \frac{\left( \frac{^{2}y}{x^{2}} \right)}{\left\lbrack {1 + \left( \frac{y}{x} \right)^{2}} \right\rbrack^{\frac{3}{2}}}$

[0480] Other embodiments of local shape feature matching compares thevolumes and surface areas of the local features that converge to aparticular skeletal entity in the final skeleton. Similarly, localmoment invariants and edge lengths for the individual features are alsoincorporated into the similarity metric.

[0481] Shell-Like and Hybrid Shapes

[0482] Shell-like and hybrid shapes produce different kinds of skeletalentities and hence require a different treatment for comparison. Globalgeometric features for these entities such as convexity and surfacecurvature will are used for similarity metrics after fitting parametricBezier surfaces to the set of voxels forming the surfaces. Finercomparisons, similar to those described for prismatic skeletons can beused for surface entities of shell-like skeletons or wireframe skeletonentities. Surfaces are discretized into a standard number of divisionsand the curvature profile as well as individual surface areas may becompared for these surfaces.

[0483] Local Emphasis and Relevance Feedback

[0484] The weights for the local geometric features such as W₁, W_(d),W_(θ) and W_(tr) (Equation 1 and 2) can be used to learn userpreferences through relevance feedback thereby reducing the semantic gapbetween the system and the user. The weights can be learned throughstandard weight reconfiguration mechanisms described in section 3.6. Inaddition, the user can also emphasize on a local feature in the 3D solidmodel or the 3D skeleton model; weights can be attributed to specificskeletal entities thereby allowing the similarity metric to give higherweight to the particular local feature while matching.

[0485] Comparison of Different Levels of Detail

[0486] The different levels of detail in the Prismatic and Shell-likeskeletons are used in the following manner. Each level of detail for adatabase model is compared with the query skeleton and the level ofdetail that yields the highest similarity value is considered as thesimilarity between the query 3D model and the particular model from thedatabase. For clustering of 3D models based on skeletal graphs all thefour levels of details for two models are compared with each other toyield the best similarity between the two models. These similarityvalues are used to cluster the database of models into groups of similarmodels. Alternatively, the user can specify particular levels of detailof all the models to be used for clustering in which case thatparticular LOD is used for clustering.

[0487] Parallelization

[0488] The graph matching step is parallelizable since this processtakes the query graph and the graphs from the database and compares eachof them individually. For the high-level graph matching step, theautomatic clusters may be used to reduce the number of comparisons.Hence, each of the prototype models for each cluster is compared withthe query graph. Subsequently, the cluster whose prototype is closest tothe query graph is used for further search. The hierarchical cluster canthus be used to reduce the search space. When the search has reached toa reasonable state where the number of models in the next cluster isreasonable (to enable real-time search), the low-level graph matchingcan be parallelized, thereby optimizing the search time.

[0489] Distinction Over Conventional Systems:

[0490] Separation of Geometry and Topology.

[0491] Hierarchical graph matching.

[0492] Inclusion of high-level geometric features in adjacency matrix.

[0493] Parallelization of graph matching in conjunction with automaticclusters.

[0494] 3.2. Multidimensional Index

[0495] With the proliferation of 3D models existing in companies, thereis an urgent requirement for indexing the database to enable efficientretrieval. Unfortunately, it is presently mission impossible for thewidely used index structures, such as B+tree and hash table, tosufficiently improve efficiency in searching 3D models, although theyhave been widely used in commercial database system. Engineers who workin 3D model similarity focus on 3D shape matching of a few models morethan shape searching of a database. Their approaches are hard to scaleto database.

[0496] Thus, embodiments presented herein have developed an indexstructure that can index the database by the similarity of the models.This structure simultaneously uses the information of each element offeature vectors to evaluate the shape similarity. An innovative andnovel method computes the overall similarity by combining thesimilarities supplied by the indexes of different features. Thecombining method coordinates the progression of the search in eachindex.

[0497] The index structure is implemented as a tree-basedmultidimensional index. Multidimensionality is important for the 3Dmodel database, because the overall similarity is defined by thecombination of these feature elements. Moreover, using feature elementas filter is inappropriate in the context of searching 3D models.

[0498] The structure of index structure is illustrated in FIGS. 49 and50. The leaf nodes contain the representations of 3D models and theinternal nodes help find a short path to the leaf nodes. Feature vectorsthat represent 3D models are grouped into leaf data nodes by theirsimilarity. Each data node is represented as a containing box thatstores the overall information of the models. Internal nodes are builtto index leaf nodes. The abstract information of each node determines ifsimilar models are possibly located in this data node. Thus, nodes arepruned and whole sub trees rooted at these nodes during a search phase.

[0499]FIG. 51A is a preliminary index structure of 39 models. FIG. 51Bshows the content in two leaf data nodes. It shows that the presentindex sufficiently groups the models by their similarity. Furthermore,algorithms are developed that efficiently use multiple multidimensionalindexes to compute the similarity. The 3D models are represented asmultiple features. Each feature has a corresponding multidimensionalindex. The overall similarity is computed with the similarity retrievedfrom each index. The efficiency of the multidimensional index isdetermined by the ability to prune the search tree. The pruning abilityis in turn depends on the search criterion, which is the radius from thequery point. The algorithm coordinates the search in each index andendeavors to minimize the radius for the search in each index. Thesealgorithms have greatly improved the pruning ability and the efficiencyof the multidimensional indexes.

[0500] In order to ensure to similar models are grouped together, a SOMis used to preprocess the feature vectors and cluster similar models.The indexes are built as bulk-loading of these clustered models. Thisgreatly improves the quality of multidimensional index, because itensures the data nodes in the index group models by similarity. If asufficient number of models are inserted in the database and corrupt thequality of index, a SOM is used again and reconstructed the index.

[0501] 3.3. Feature Vector Comparison

[0502] Current search systems usually use single type features that arenot sufficiently capture the intrinsic characteristics of 3D models.Even a few techniques use combinational feature vectors; the internalrelationships between these features have not been fully understood. Alinear combination is simple but commonly used to compute the overallsimilarity based on that of each feature, irrespective of theirrelationships.

[0503] Correspondingly, in embodiments of this invention, more andintrinsic feature vectors are extracted to capture the importantcharacteristics of 3D models. Better understanding the relationships offeature vectors enabled one to develop more advanced methods to computethe overall similarity based on those defined on individual features.

[0504] The present invention uses a system that is a feature-based 3Dmodel search system. It uses low-level features instead of high-levelkeywords to represent the complex geometry. The original 3D models wereprocessed which generated different models with different level ofdetails. From these models, the statistical, spatial, and temporalinformation are extracted as features that representing 3D models. FIG.52 is the feature vectors that are currently used in embodiments of thepresent system. These feature vectors are the numeric finger-prints ofthe original 3D models. The comparison and searching of 3D models areapplied on these finger-prints, because the searching the originalmodels is intractable.

[0505] Considering feature elements as coordinates, a 3D model isfurther represented as a point in the multidimensional feature spacewhich is illustrated in FIG. 53 using a two dimensional space as anexample.

[0506] Considering a model as a data point in the feature space,different distance defined on the coordinates is used to measure thesimilarity of the models. For example, the Euclidean distance L2,Maximum metric L∞, and Manhattan metric L1 are used. The equations tocompute these distances are listed as follows.

[0507] Based on these similarity definitions, the similarity measure foreach feature is computed. Not like feature-based image search, some ofthe features in the system are not totally independent. Therefore, whenthe overall similarity based on feature similarities is computed, morecomplex combination methods besides linear combination are used. Thesemethods include quadratic, cubic and even a complex function representedby neural network. Furthermore, graph matching is used with togetherwith feature based similarity search. The feature representation isdescribed here and graph representation is introduced in graph matchingsection.

[0508] Instead of using a one-step searching method, which uses all thefeature vectors simultaneously to search the database, the search issimulated as a decision tree. Based on cognitive psychology research,some selective features are included, such as aspect ratio and symmetricinformation. In other embodiments, some other more advanced features canbe used. Furthermore, features that have been previously used can beused subsequently.

[0509] The advantages of this retrieval system are: (1) it is anobjective approach, since all the features are intrinsic values that arecomputed by the system objectively. The representation of the models inthe databases is determined by the feature-extraction algorithms. Therepresentations of CAD models in the system are unbiased by any user.Thus, the system has a neutral version of features for all users. (2) Itis easy to automate, since all of the information is objective.Algorithms can be developed which are computed from the CAD models,without any involvement from the users. (3) It is more accurate toquantify the degree of the similarity than the keyword-based retrievalsystem. Since each feature is in a term represented by a scale number(usually 0-1), the similarity can be easily characterized by somemathematical operation. As long as the system has extracted enoughinformation from he CAD models, the system will successfully retrievethe similar models.

[0510] The main disadvantages lie in the difficulty in defining thefeature vectors that are suitable for filtering all dissimilar modelswhile retaining all the similar ones. However, a multi-step searchstrategy (described in section 3.8) adopted with embodiments of thisinvention allow the user to experiment with different feature vectors toachieve the desired set of models.

[0511] 3.4. Clustering Techniques

[0512] Embodiments of the present invention allow the user to easilybrowse through different models in the database to find parts similar tohis/her requirements. A portion of the visual query interface isdedicated to facilitate this functionality. The interface can bevisualized to be working in two different scenarios (see FIG. 54). Inthe first case the user may only have formed a mental idea of the 3Dmodel and accordingly wants to browse through the parts available in thedatabase and pick the relevant parts. In order to make this searchefficient, a hierarchical part cluster map of all the models in thedatabase is provided which is updated dynamically with the addition ofnew models to the database. Each part cluster in a higher level clusterwill point to a number of clusters in the next level and so forth. Inthe other scenario, the user may input a query model from the sketchinginterface, and all the parts that are found similar to the query areretrieved. Each of the retrieved models shows a link to the cluster thatit belongs, thereby allowing further browsing. The corresponding linkcan be followed to browse the cluster to which the model belongs.

[0513] Clustering algorithms are extensively used in the field ofknowledge discovery and data-mining, the most important of which arehierarchical clustering, k-clustering and Self-Organizing Maps (SOMs).Of these different algorithms, the SOMs are the easiest to implement andalso allow for easy visualization and interpretation of the clusters. ASOM is a neural network algorithm that is based on unsupervisedlearning. It consists of a layer of computational units, often calledneurons that adapt themselves to a population of input patterns. Uponpresenting an input pattern the neuron that is closest to this patternis determined (called the winning neuron) and then updated to includethe new pattern, while some of the neighboring nodes are updated to movecloser to the winning neuron. The SOM has properties of both vectorquantization and vector projection algorithms, thereby allowingprojection of data in higher dimensions to lower (typicallytwo-dimensions) which is amenable for visualization. This feature isadvantageous with embodiments of this invention because it assists indeveloping a useful visual interface for the user to browse through themodels. A SOM also allows dynamic updating of the cluster map withaddition of new parts to the database.

[0514] Traditionally SOMs have been used only with feature vectors. Onlyrecently, have there been attempts to use the SOMs with more powerfuldata structures such as graphs and trees for use with patternrecognition tasks. These attempts have used a graph distance metricbased on graph edit distance which is used update the neurons in theSOM. Conversely, the embodiments of the presenting invention uses acombination of (a) feature vectors obtained from the 3D model and (b)graph distance measure distances obtained from graph comparison. Hence,the neuron is designed to contain all the feature vectors as well as arepresentative graph of the 3D skeleton (see FIG. 55). The SOM isupdated to show the new cluster map (1) with different similaritydefinitions configured by the user (2) with addition of new models tothe system. This interface is useful in quickly evaluating the (1)similarity metrics proposed in this invention and (2) metrics proposedin prior techniques for comparing 3D models. The distance metric betweentwo neurons can be any of the standard vector distances such asEuclidean, Manhattan distance, etc.

[0515] Distinction from Prior Techniques:

[0516] 1. Inclusion of graph properties in SOM neuron data structure.

[0517] 2. Use of SOM for clustering 3D models based on skeletal graph.

[0518] 3. User interaction in forming similarity definition forclustering.

[0519] 3.5. Cluster Map Interface

[0520] The cluster map interface is the main visual output interface forthe search system. The interface works in two scenarios 1) Automatic 2)Interactive. FIG. 56 illustrates the two different paths for the twosituations. In the automatic mode, the database models are classifiedinto various clusters automatically, based on the default similaritymetrics defined in the system. The cluster map (FIG. 57) presentsrepresentative 3D models to the user on a part cluster window allowingthe user to pan, zoom and rotate the model with buttons and mousemotion. The field to the right of each picture shows a bar graph of thenumber of models present in the particular cluster. Further browsing ina cluster is allowed by clicking the “Similar Models” button. In theinteractive mode the user 1) communicates his similarity measure byreconfiguring the feature vectors and 2) also browses the hierarchicaldatabase to search for the model he has in mind. The part cluster map isupdated accordingly and presented to the user through the cluster mapinterface.

[0521] The cluster map interface is also used to represent the similarmodels retrieved after submitting a search query (FIG. 58). Each of themodels, retrieved from similarity query, provides a link to the clusterit belongs to, through the “Similar Models” button. The degree ofsimilarity of each retrieved model is also presented to the user as apercentage measure. Each model is also provided with a slid bar to giverelevance feedback to the system for neural network learning.

[0522] 3.6. Relevance Feedback

[0523] The retrieval system only measures the similarity in terms of thelow level feature vectors. However, a user's similarity is based on hisperception. Since the 3D models are complex objects it is challenging toextract features that accurately represent the 3D models. There is a“semantic gap” between the similarity definition of low level featurevectors and the high level 3D models.

[0524] Conventional 3D shape search systems search models are based onpredefined similarity measures and one-shot mechanism. The semantic gapof similarity between users and system seriously corrupts theperformance of these conventional systems.

[0525] However, relevance feedback is an important way to improve thecommunication between users and the system. It is a bridge between thetwo semantic planes on which the system and users similarity definitionslie, respectively. The relevance feedback greatly reduces the burden forthe users to accurately construct a query and to choose the features forsimilarity measure. In addition, it enables the search system to learnfrom the communication and reconfigure the system based on thecommunications.

[0526] An example procedure consists of the following steps: (a) whenthe system retrieves some models based on the query example, it ordersthem by similarity measure of feature vectors and presents them to theuser. (b) Users classify them into relevant and irrelevant groups andinform the system. (c) The system, in turn, configures itself to respectthe user's preference and redo the search. (d) This process iterates forsometime until either the user is satisfied or the system thinks that itcannot make it better.

[0527]FIG. 59 illustrates the procedure of relevance feedback. Differentmethods to use the information in user's relevance feedback can also beimplemented and are intended to fall within the generous scope of thepresent invention. The methods are based on computational geometry andsimulating social behavior, such as particle swarms.

[0528] The presented methods enable communication of users and thesystem, thus bridging the semantic gap of similarity. In addition,algorithms are developed that reconstruct a query vector and reconfigurethe system based on the relevance feedback.

[0529] Search results are presented on the interface so that a user cantell the system what the relevant and irrelevant models are from his/herpoint of view.

[0530] Algorithms are developed that reconfigure the search system,based on the information of relevance feedback. This is important forany successful search system, because different users may have differentperception about the similarity of 3D models. Initially, the system isconfigured to reflect the similarity of most users, for example, it canfind models that 90% users think similar. When a user is using thesystem for searching, his/her relevance feedback is used to finereconfigure the system. From feedback, the system can use featurevectors that the user are more interested in and reconfigure the systemto reflect user's preference. Also, the system can reconstruct the queryvector based on the initial query vector and the relevance feedback.

[0531] 3.7. Learning Agents

[0532] Although relevance feedback is an effective way to bridgesemantic gap, it is time consuming. Even for the same model, if usersubmits as query model again, the time-consuming relevance feedback hasto be done again.

[0533] Thus, techniques for embodiments of this invention implementartificial intelligence—neural networks—to learn from user's relevancefeedback. This technique enables the customization of the system todifferent users and input query models without changing the content ofdatabase and index structures. It is a layer above the index forlearning and customization. FIG. 600 illustrates one such method.

[0534] Neural networks are deployed to learn and identify the patternsin a user's relevance feedback. The key point is to implement networksystems that can be trained fast, such as a neural network. Amulti-layer feed-forward neural network system was used for someembodiment of this invention. Algorithms from social behavior—particlesystem, and back-propagation can be used to initially train the neuralnetwork.

[0535] A learning agent is used to quickly reconstruct the query vectorthat better represents a particular user's intention. Because of thesemantic gap, it is difficult for a user to submit a query with lowlevel feature elements. Query by example may only let a user give arough idea about his/her intention. Therefore, the query vector isrefined to better interpret a user's intention. Relevance feedback thata user has done is used as a training example, if the user activates thelearning agent. The goal is to directly reconstruct a query vector basedon a user's searching history and currently submitted query. The keypoints include detecting the maturity of neural network, and avoidingover-fitting.

[0536] A learning agent can also be used to reconfigure the searchsystem, i.e., reconfigure the weights of the feature elements. If astable pattern is identified, then the system can be reconfigured toreflect a user's view about the relative importance of features. Thegoal here is also to save the time for relevance feedback. Theadvantages of introducing learning agent are: a) to separate the queryreconstruction and weight reconfiguration during the relevance feedback,b) to customize the system for different users and/or different queriessubmitted by the same user.

[0537] What makes the neural network an important contribution is thatit can customize the system to both users and the query input. Thefollowing is an example that shows why the weight setting may depend oninput in some cases.

[0538] Suppose the feature vector consists of two elements. One is F1and the other is F2. FIG. 61 shows feature vectors in the database onthe left side of the table. The user's query vectors are on the rightside of the table. The arrows show the matching between the user's queryinputs and those in the database. This matching can be discovered by theuser's relevance feedback.

[0539] Based on the first three relevance feedback, it is reasonable toreconfigure the weights as W₁ ⁽¹⁾=W₁ ⁽⁰⁾ and W₂ ⁽¹⁾=3×W₂ ⁽⁰⁾ However,based on the second three relevance feedbacks, the system isreconfigured to W₁ ⁽¹⁾=W₁ ⁽⁰⁾ and W₂ ⁽¹⁾=10×W₂ ⁽⁰⁾. In other words, thereconfigurations depend on the scope where F1 lies in. If one weight issimply assigned, the accuracy of the system can be far from satisfactoryfor some of the models, although it may work for some others.

[0540] 3.8 Multi-Step Search Process

[0541] A true search system should allow a user to find a model whichhe/she is looking for, instead of presenting a set of models based on apredefined similarity metric. Towards this end the embodiments of thisinvention incorporate a multi-step search strategy to allow the user tosearch for a part using different matching criteria. For example, a usercan search the database using skeletal graphs in the first step of thesearch, which will retrieve topologically similar models. Subsequently,a different metric, say the total volume, can be used to filter out theretrieved models. This allows the user to use Boolean operations for thesearch criteria, thereby retrieving a relevant part quickly.

[0542] The multi-step approach is the first of its kind proposed for a3D shape search system in general and for the engineering domain inparticular.

[0543] Embodiments of the present invention's system allow incorporatingsimilarity invariants as well as Skeletal graphs for the multi-stepsearch. One instance of such a multi-step search process is presented inFIG. 62:

[0544] a. Various feature vectors (viz. Moments invariants, Eigenvaluesof skeletal graphs, Geometric parameters) are used to reduce searchspace (Database with N models) to small number, M₀, where M₀<N.

[0545] b. Skeletal Graph-Invariants (<Degree of Nodes>, <Number of L, Eand C>, <Number of L-L, L-E, E-E connections>, etc.) used to furtherreduce search space, to say M₁, where M₁<M₀.

[0546] c. Inexact Graph Matching is performed to further reduce thesearch space and for ordering the search results on this smaller set ofmodels, M₂<M₁.

[0547] d. User interacts with system in the following steps to reducesearch space (if necessary)

[0548] i. Specify Invariants-based criteria for reducing search space.

[0549] ii. Use Boolean operations on search criteria.

[0550] iii. Specify Similarity Threshold criteria for final results.

[0551]FIG. 63 depicts a particular search result from multi-steprefinement where the initial query was posed using the skeletalgraph-based similarity and the results were refined using momentinvariants to retrieve the relevant models.

[0552] Although specific embodiments have been illustrated and describedherein, those of ordinary skill in the art will appreciate that anyarrangement calculated to achieve the same purpose can be substitutedfor the specific embodiments shown. This disclosure is intended to coverall adaptations or variations of various embodiments of the invention.It is to be understood that the above description has been made in anillustrative fashion only. Combinations of the above embodiments, andother embodiments not specifically described herein will be apparent toone of ordinary skill in the art upon reviewing the above description.The scope of various embodiments of the invention includes any otherapplications in which the above structures and methods are used.Therefore, the scope of various embodiments of the invention should bedetermined with reference to the appended claims, along with the fullrange of equivalents to which such claims are entitled.

[0553] It is emphasized that the Abstract is provided to comply with 37C.F.R. § 1.72(b), which requires an Abstract that will allow the readerto quickly ascertain the nature and gist of the technical disclosure. Itis submitted with the understanding that it will not be used tointerpret or limit the scope or meaning of the claims.

[0554] In the foregoing Detailed Description, various features aregrouped together in single embodiments for the purpose of description.This method of disclosure is not to be interpreted as reflecting anintention that the claimed embodiments of the invention require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. The following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separate preferred embodiment.

What is claimed is:
 1. A method for searching, comprising: receiving athree dimensional object; searching one or more data stores with thethree dimensional object as a first search query; presenting resultsfrom the search, wherein the results include an answer set; receivingmodifications or relevance information for one or more items in theanswer set; and re-searching the one or more data stores with themodified or relevance information associated with the one or more itemsas a second search query.
 2. The method of claim 1 further comprising,converting the three dimensional object into a graph skeleton defining agraph data structure, wherein the graph data structure is the firstsearch query.
 3. The method of claim 1 wherein receiving the threedimensional object further includes interactively permitting the threedimensional object to be sketched.
 4. The method of claim 1 wherein thereceiving the three dimensional object further includes presenting alist of three dimensional models and permitting the three dimensionalobject to be formed from selective ones of the list of three dimensionalmodels.
 5. The method of claim 1 wherein the presenting the resultsfurther include grouping selective portions of the one or more items inthe answer set into related clusters.
 6. The method of claim 1 furthercomprising, receiving one or more filters which constrain the first orsecond search queries.
 7. The method of claim 1 wherein the re-searchingfurther includes identifying in the relevance information for the one ormore items information that identifies selective ones of the items thatare more relevant to the first search query than selective other ones ofthe items.
 8. A method of searching, comprising: receiving a twodimensional object; mapping the two dimensional object to a threedimensional representation; searching one or more data stores with thethree dimensional representation as a first search query; and presentingone or more items in an answer set that is responsive to the firstsearch query of the one or more data stores.
 9. The method of claim 8wherein the mapping further includes: representing the two dimensionalobject as a two dimensional skeleton; converting the two dimensionalskeleton into a three dimensional skeleton; and representing the threedimensional skeleton as a three dimensional graph structure, wherein thethree dimensional graph structure is used as the first search query. 10.The method of claim 8 further comprising: receiving relevanceindications for a selective number of the one or more items in theanswer set; and searching the one or more data stores with the selectivenumber of the one or more items and the relevance indications as asecond search query.
 11. The method of claim 10 retaining the relevanceindications as preferences for subsequent search queries received andprocessed, where the retained relevance indications are used as filtersto subsequent first queries.
 12. The method of claim 8 furthercomprising organizing the answer set as a plurality of related clusters,wherein each related cluster includes a selective number of the one ormore items.
 13. The method of claim 8 wherein the mapping furtherincludes: converting the two dimensional object into a two dimensionalskeleton; generating candidate three dimensional vertices for each oftwo dimensions of the two dimensional skeleton; generating candidatethree dimensional edges from the candidate three dimensional vertices;creating candidate three dimensional faces from the three dimensionaledges on a same surface; creating one or more three dimensional objectsfrom the candidate three dimensional faces; and associating the one ormore three dimensional objects with the received two dimensional objectas the three dimensional skeleton.
 14. A method of presenting searchresults, comprising: presenting three dimensional objects within one ormore related clusters as an answer set in response to a first searchquery; interactively receiving relevance information associated withselective ones of the three dimensional objects or selective ones of therelated clusters; and interactively issuing one or more second searchqueries based on the received relevance information to remove a numberof the three dimensional objects from the answer set and to reduce anumber of the related clusters from the answer set.
 15. The method ofclaim 14 wherein the presenting further includes presenting the threedimensional objects within the one or more related clusters as answerset in response to the first query where the first query is a threedimensional search request derived from an initial two dimensionalsearch request.
 16. The method of claim 14 wherein the receiving of therelevance information further includes receiving selective featurevectors as a portion of the relevance information, wherein the selectivefeature vectors define selective ones of the three dimensional objectsin more than one dimension.
 17. The method of claim 16 wherein thereceiving of the relevance information further includes defining theselective feature vectors based on lower level constructs associatedwith the selective ones of the three dimensional objects.
 18. The methodof claim 14 wherein the presenting further includes presenting theanswer set in response to the first search query where the first searchquery is obtained by interactively traversing a hierarchy of availablethree dimensional models and selecting one of the available threedimension modes as the first search query.
 19. The method of claim 14wherein the presenting further includes forming the one or more relatedclusters using a Self Organizing Map (SOM) clustering technique.
 20. Themethod of claim 19 wherein the presenting further includes using a SOMwhere feature vectors for each of the related clusters include acombination of a cluster feature vector for the three dimensionalobjects included in each cluster and graph distance measures obtainedfrom comparisons of components associated with each of the threedimensional models included in each cluster.
 21. A method to form athree dimensional search request, comprising: deriving a topologyassociated with a three dimensional object; deriving local geometriesassociated with components of the three dimensional object; and formingthe topology and the local geometries into a search request.
 22. Themethod of claim 21 wherein the deriving the topology further includesobtaining relationships between geometric entities of the threedimensional object.
 23. The method of claim 21 wherein the deriving thelocal geometries further includes obtaining local geometriccharacteristics for each geometric entity included within the threedimensional object.
 24. A method of populating a searchable threedimensional data store, comprising: reducing three dimensional models tothree dimensional skeletons; deriving for each three dimensionalskeleton a topology and local geometries associated with each geometricentity defined in that skeleton; creating a feature vector for eachthree dimensional skeleton and its associated topology and associatedlocal geometries; creating associations between each of the threedimensional skeletons and their associated feature vector; and housingeach association in a three dimensional data store.
 25. The method ofclaim 24 wherein the deriving further includes: representing thetopologies as relationships between each of the geometric entities forthe skeletons; and representing the local geometries as local geometriccharacteristics for each of the geometric entities for the skeletons.26. The method of claim 24 further comprising interfacing a high-levelmatcher with the three dimensional data store, wherein the high-levelmatcher evaluates topology information of the feature vectors inresponse to a search request issued to the three-dimensional data store.27. The method of claim 26 further comprising interfacing a low-levelmatcher with the three dimensional data store, wherein the low-levelmatcher evaluates local geometry information of the feature vectors inresponse to the search request.
 28. The method of claim 27 furthercomprising interfacing the high-level matcher with the low-level matcherto resolve candidate feature vectors which match the search request. 29.The method of claim 28 further comprising determining the candidatefeature vectors which are responsive to the search request based on asubstantial match between a search vector associated with the searchrequest and the candidate feature vectors.
 30. The method of claim 28further comprising determining the candidate feature vectors which areresponsive to the search request based on a predefined thresholdpercentage of a match between a search vector associated with the searchrequest and the candidate feature vectors.
 31. The method of claim 24wherein the deriving further includes obtaining the local geometries ascharacteristics associated with geometric entities of the skeletons,wherein the characteristics includes at least one or more of entityvolumes, entity volume distributions, and thickness distributionsthrough distance.
 32. A method for searching, comprising: receiving atwo or more dimensional object; converting the two or more dimensionalobject into a three dimensional skeleton; receiving a selection for apart of the three dimensional skeleton; converting the part into afeature vector; and searching a data store for the feature vector. 33.The method of claim 32 further comprising presenting candidate partsreturned from the data store in response to the feature vector.
 34. Themethod of claim 33 further comprising presenting candidate threedimensional models which have the candidate parts.
 35. A method ofsearching, comprising: receiving a two or more dimensional object;reducing the two or more dimensional object to a three dimensionalskeleton; deriving from the two or more dimensional object textinformation; forming a feature vector from topology and local geometriesassociated with the three dimensional skeleton and from the textinformation; and submitting the feature vector as a search request to adata store.
 36. The method of claim 35 wherein the deriving furtherincludes associating the text information with at least one of amaterial identifier, a supplier identifier, a plant identifier, adesigner identifier, a color identifier, a pattern identifier, and adate of creation identifier.
 37. The method of claim 35 furthercomprising presenting an answer set associated with the search requestthat is clustered by related search results.
 38. A method of performinga search, comprising: acquiring a relevance preference for a threedimensional search request; modifying the three dimensional searchrequest with the relevance preference; and submitting the threedimensional search request to one or more data stores.
 39. The method ofclaim 38 further comprising: presenting one or more clusters having oneor more three dimensional models from an answer set returned from theone or more data stores in response to the three dimensional searchrequest; receiving additional relevance preferences for the answer set;modifying the relevance preference with the additional relevancepreferences; and associating the modified relevance preference with asearcher that is associated with the search request.
 40. The method ofclaim 39 further comprising: receiving a new three dimensional searchrequest from the searcher; modifying the new three dimensional searchrequest with the modified relevance preference associated with thesearcher; and submitting the new three dimensional search request to theone or more data stores.
 41. A search system, comprising: a threedimensional query interface; a skeleton converter that interacts withthe query interface; and a feature vector constructor that interactswith the skeleton converter; wherein the query interface receives searchrequests for objects in two or more dimensions and the objects arerepresented by the skeleton converter as three dimensional graphicaldata structures, the feature vector constructor translates the graphicaldata structures into feature vectors that are used as search requests toone or more data store, and wherein the feature vectors include topologyand local geometric information derived from the three dimensionalgraphical data structures.
 42. The search system of claim 41 furthercomprising: a high-level matcher that interfaces to the one or more datastores which compares the topology information portions of the featurevectors against candidate feature vectors indexed in the one or moredata stores; and a low-level matcher that interfaces to the one or moredata stores which compares the local geometric information portions ofthe feature vectors against the candidate feature vectors indexed in theone or more data stores.
 43. The search system of claim 41 wherein thequery interface presents answer sets associated with results of thesearch requests as one or more related clusters.
 44. The search systemof claim 43 wherein the query interface permits one or more searchresults included within the one or more clusters to be assignedrelevance information.
 45. The search system of claim 44 wherein theassigned relevance information is assigned automatically by the queryinterface or assigned via manual selections made by a searcherinterfacing with the query interface.
 46. The search system of claim 41wherein the query interface receives the objects based on interactionswith a searcher that interactive creates the objects within the queryinterface.
 47. The search system of claim 41 wherein the query interfacereceives the objects based on interactive selections made by a searcherthat traverses hierarchies of model objects.
 48. The search system ofclaim 41 wherein the query interface receives the objects directly froma searcher via a paste operation or a file import operation.
 49. Thesystem of claim 41 wherein the query interface permits a searcher tointeractively select parts associated with the graphical datastructures, and wherein the feature vector constructor uses the parts toform the feature vectors.
 50. A search system, comprising: a threedimensional data store; an indexer; and a comparer; wherein the indexerassociates feature vectors with three dimensional graph data structuresand the feature vectors include topology and local geometric informationfor entities of each graph data structure, and wherein the featurevectors and the graph data structures are associated with one anotherand housed in the data store, the comparer receives search requests assearch vectors and searches the data store for candidate feature vectorsthat at least partially match the search vectors and returns theassociated graph data structures as answer sets for the search vectors.51. The search system of claim 50, wherein the comparer furtherincludes: a high-level matcher that matches topologies associated withthe search vectors with the topology information for the candidatefeature housed within the data store; and a low-level matcher thatmatches local geometries associated with the search vectors with thelocal geometric information for the candidate feature vectors housedwithin the data store.
 52. The search system of claim 50 wherein thecomparer is configured to determine the at least partial match based ona preconfigured percentage of components of the search vectors thatmatch candidate components of the candidate feature vectors.
 53. Thesearch system of claim 50 wherein the comparer is configured todetermine the at least partial match based on a percentage of componentsof the search vectors that match candidate components of the candidatefeature vectors, and wherein the percentage is part of the searchvectors or included separately with the search vectors for consumptionby the comparer.
 54. The search system of claim 50 wherein the searchvectors define parts of a three dimensional object.
 55. The searchsystem of claim 50 wherein the search vectors define complete threedimensional objects.
 56. A query interface system, comprising: a threedimensional search constructing interface; a primitive data store ofthree dimensional primitives; and a cluster interface; wherein thesearch constructing interface is interacted with to assemble one or moreof the three dimensional primitives from the primitive data store andform a three dimensional search query, and wherein the cluster interfaceorganizes search results associated with results of search query intogroupings of similar results.
 57. The query interface of claim 56,wherein the cluster interface uses preconfigured or searcher definedcriteria to organize the groupings.
 58. The query interface of claim 56further comprising a vector choice interface that permits a searcher todynamically modify the search query by placing relevance information onone or more of the search results or placing the relevance informationon portions of a single search result and resubmit the modified searchquery.
 59. The query interface of claim 56 wherein the searchconstructing interface permits one or more new three dimensionalprimitives to be defined by a searcher and housed in the primitive datastore.
 60. The query interface of claim 56 wherein the searchconstructing interface permits a searcher to import or past a two ormore dimensional object into the search constructing interface andreduces the object into derived primitives that are matchedautomatically to a selective number of the three dimensional primitiveshoused in the primitive data store.
 61. A search constructor system,comprising: a three dimensional candidate constructor; and a featurevector constructor; wherein the three dimensional candidate constructorreceives two dimensional objects and converts them as three dimensionalobjects represented by three dimensional graph data structures, andwherein the feature vector constructor derives topology and localgeometric information from the three dimensional graph structures toconstruct search vectors for a three dimensional search.
 62. The searchconstructor system of claim 61, wherein the three dimensional candidateconstructor generates three dimensional candidate vertices from verticesincluded in the two dimensional objects, generates three dimensionalcandidate edges from the three dimensional candidate vertices,constructs three dimensional candidate faces from the three dimensionalcandidate edges, and constructs the three dimensional objects from thethree dimensional candidate faces.
 63. The search constructor system ofclaim 62, wherein the three dimensional candidate constructor matchesloops and the three dimensional candidate vertices to form geometricentities.
 64. A geometric graph data structure, implemented in acomputer readable medium, the graph data structure used to define threedimensional objects for purposes of search and retrieval, the graph datastructure comprising: one or more geometric entities; a topology matrixthat defines topology connections between the one or more geometricentities; and one or more geometric shapes having geometric featuresassociated with one or more geometric entities and wherein eachgeometric feature can be reduced to a single one of the geometricentities; wherein the geometric graph data structure is generated froman object having two or more dimensions and wherein a feature vector isderived for the geometric data structure, wherein the feature vectorthat includes portions of the topology matrix and the features.
 65. Thegraph data structure of claim 64, wherein each of the entities aredefined by a set of edges, a set of loops, a set of surfaces, and a setof holes.
 66. The graph data structure of claim 64, wherein each of theone or more shapes further include a shape topology matrix that definestopological connectivity between the geometric features of each shape.67. A feature vector implemented in a computer readable medium, thefeature vector defining searchable attributes of a three dimensionalobject and used for generating search requests or for populating asearchable data store with as search keys associated with the threedimensional objects, the feature vector comprising: topology informationthat defines the connectivity and relationships between geometricentities associated with a three dimensional object; and localgeometries associated with characteristics of each of the geometricentities.
 68. The feature vector of claim 67, wherein the feature vectoris associated with a single part of the three dimensional object. 69.The feature vector of claim 67, wherein the feature vector is associatedwith one or more parts of the three dimensional object but not thecomplete three dimensional object.
 70. The feature vector of claim 67,wherein the feature vector is used to perform a search against a threedimensional data store having a plurality of instances of the featurevector, wherein each of the instances represents a different threedimensional object instance or a different part of the different threedimensional object instances.
 71. The feature vector of claim 67,wherein the feature vector is used to determine a partial match againstone or more of the plurality of instances of the feature vector housedin the three dimensional data store.
 72. A method of searching,comprising: selecting features associated with an object represented inmore than two dimensions; selectively weighting the features; andsubmitting the selectively weighted features as a search request. 73.The method of claim 72 wherein selecting the features further includesselecting between graph data structure elements representing the objector selecting portions of a feature vector describing the object.
 74. Themethod of claim 72 wherein selectively weighting further includesweighting local or global features associated with the object.